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EVALUATION 


This  work  described  in  this  report  and  performed  during  this 
effort  has  established  that  virtual  machines  can  be  a significant 
asset  as  a software  engineering  tool  in  a sophisticated  programming 
environment.  This  effort,  performed  on  the  H6180,  has  also 
demonstrated  that  VMM's  in  the  RADC  Programming  Environment  are 
feasible  with  several  performance  tradeoff  factors. 

The  results  from  this  effort  are  extremely  Important  in  future 
efforts  involving  distributed  processing  as  outlined  In  RADC 
Technology  Plan  (TPO  V).  In  a system  of  logically  distributed 
processing,  certain  systems  In  the  network  will  be  performing 
specialized  functions  on  behalf  of  the  other  member  systems.  The 
role  of  virtual  machines  In  isolating  these  functions  will  be 
critical  to  insure  optimal  performance.  As  these  functions  are 
Integrated,  virtual  machines  can  again  provide  tne  control  techniques 
required  to  insure  compatibility  among  all  systems. 

This  effort  has  also  demonstrated  the  role  required  by  virtual 
machines  in  hardware/software  tradeoffs.  Decreasing  hardware  costs 
will  induce  efforts  to  perform  many  current  software  functions  In 
hardware.  Virtual  machines  will  play  an  Important  role  In  this 
transition  which  can  lead  to  a reduction  in  software  costs. 


j/frtryrdr  / 

(D  A.  LIUZZI 
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SECTION  I 

INTRODUCTION 


GOALS  AND  HISTORY 

This  document  describes  The  Virtual  Machine  Monitor  Performance 
Analysis.  A Virtual  Machine  Monitor  (VMM)  is  an  operating  system  which 
executes  on  the  native  hardware  and  allows  other  (standard)  operating 
systems  to  execute  in  an  environment  much  like  the  environment  an 
operating  system  provides  for  user  programs.  These  sub -operating 
systems  are  called  virtual  machines,  or  VMs.  A VM  looks  to  its  con- 
tained operating  system  much  like  the  actual  machine  looks  to  the  VMM. 

While  the  VMM  is  aware  of  the  actual  hardware  complement  available,  a 
VM  is  aware  of  only  the  hardware  (or  simulated  hardware  in  the  case  of 
some  peripherals)  provided  to  it  by  the  VMM.  By  providing  these  virtual 
environments,  a single  hardware  base  can  be  used  by  a number  of  differ- 
ent standard  and  nonstandard  operating  systems. 

The  values  of  such  an  arrangement  are  many,  particularly  in  an  environ- 
ment which  engages  in  research  on  operating  systems.  A standard 
production  service  can  be  provided  concurrently  with  an  experimental 
or  low-use  service.  Thus,  the  need  for  off-hour  scheduling  of  machine 
time  and  interrupted  service  for  boot  and  re-boot  can  be  reduced. 
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VMM  research  in  Honeywell  and  the  industry  (particularly  by  IBM:  VM/370) 
has  a long  history.  The  Honeywell  activity  began  in  the  early  1970's  as  a 
strategy  for  product  line  unification  and  to  provide  more  flexible  internal 
use  of  computer  equipment. 

Interest  at  Rome  Air  Development  Center  (RADC)  began  in  approximately 
1974  with  an  intense  look  at  the  security  aspects  of  providing  isolated 
environments  for  operating  systems  separated  from  each  other  by  hardware 
enforced  boundaries.  This  led  to  a study  of  the  GCOS  Environment 
Simulator  or  GCOS  encapsulation  on  Multics.  This  tool,  though  not  as 
powerful  conceptually  as  the  VMM,  is  currently  in  use  at  RADC. 

Business  decisions  made  by  Honeywell  mandated  that  the  VMM  remain  in 
the  experimental  stages,  but  low  level  research  continued.  In  1976,  RADC 
entered  into  negotiations  with  Honeywell's  Federal  Systems  Operations  to 
procure  a VMM  for  further  study  at  RADC.  This  led  to  the  installation  of 
the  VMM  at  RADC  in  1977  and  subsequently  to  the  effort  described  in  this 
document. 

Honeywell's  Systems  and  Research  Center  analyzed  the  performance  of 
this  hardware/ software  package  for  RADC  to  locate  the  system  bottlenecks 
known  to  exist  and  to  help  RADC  plan  a strategy  for  the  evolution  of  VMM 
research.  The  findings  of  that  task  constitute  a detailed  analysis  of  the 
performance  of  the  Honeywell  VMM,  some  suggestions  for  improvement 
should  RADC  desire  to  continue  the  experiment,  the  results  of  a model  of 
performance  constructed  by  BGS  Systems,  Inc. , and  recommendations 
for  future  research. 
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The  remainder  of  this  section  contains  background  and  summary  inform- 
ation. Section  II  contains  a detailed  description  of  the  means  used  to 
collect  performance  data  and  an  analysis  of  that  data.  Section  III  contains 
plans  for  the  evolution  of  the  VMM  and  a projection  of  the  role  of  VMM 
research  in  future  computer  products.  The  final  section  details  recom- 
mendations for  extending  the  VMM  capability  and  continuing  research  in 
this  area.  Appendixes  contain  the  computer  listings  of  the  job  scripts  and 
monitoring  tools  used  to  meter  performance,  as  well  as  the  text  of  the 
interim  report. 

SUMMARY  OF  RESULTS 

Results  based  on  live  data  experiments  and  the  BEST/1,  analysis  show 

tm 

the  following: 

• VMM  overhead  has  its  greatest  effect  on  work  loads  exhibiting 
an  intermediate  amount  of  I/O  activity  (15  to  35  connects  per 
second  of  processor  busy  time). 

• For  work  loads  with  a small  amount  of  I/O  activity,  VMM  has 
only  a minor  effect  on  performance.  With  large  amounts  of 
I/O,  contention  at  the  I/O  devices  is  the  limiting  factor. 

• In  GCOS,  the  VMM  overhead  was  determined  to  be  in  the  range 
of  15  to  28  percent  depending  upon  the  I/O  mix.  This  results  in 
an  overhead  of  3.  5 percent  of  processor  busy  time  and  4.  5 msec 
of  overhead  per  connect. 

• In  Multics,  the  VMM  overhead  was  determined  to  be  between  13 
and  60  percent.  These  higher  figures  are  due  to  the  increased 
dependence  of  Multics  on  I/O  activity  to  process  page  faults. 
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Based  on  these  figures  and  the  insufficiency  of  data  (particularly  in  the 
Multics  case),  it  can  be  observed  that  VMM  overhead  is  directly  related 
to  I/O  activity.  Thus,  any  further  work  on  the  performance  of  this  VMM 
should  concentrate  on  I/O  monitoring  and  speedup. 

PERSONNEL 

The  people  involved  in  this  effort  were: 

• Mr.  Ray  Liuzzi- -contract  monitor  for  this  project.  He  played 
a valuable  role  by  providing  technical  direction  and  leadership 
in  virtual  machine  research.  His  interest  was  instrumental  in 
promoting  the  potentials  of  VMMs  for  Air  Force  applications. 

• Stanley  C.  Vestal- -formerly  of  the  Systems  and  Research  Center, 
now  a scientist  with  the  Corporate  Computer  Sciences  Center. 

Mr.  Vestal  served  as  Principal  Investigator  for  the  contract. 

• Dr.  Harold  Schwenk- -formerly  with  Honeywell  Information 
Systems  and  now  the  President  of  BGS  Systems,  Inc.,  a sub- 
contractor on  this  effort.  Dr.  Schwenk  was  a co-designer  of 
the  VMM. 

• Tom  Krocak- -Honeywell  Computer  Network  Operations.  He  was 
responsible  for  the  analysis  of  GCOS  performance. 

Additional  consulting  was  obtained  from: 

• Russ  McGee- -Honeywell  Information  Systems.  He  was  the  princi 
pal  designer  of  the  VMM  and  leader  of  the  VMM  program  in 
Honeywell. 
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• Dr.  Robert  Goldberg --formerly  of  Honeywell  Information  Systems 
and  now  with  BGS  Systems,  Inc.  Dr.  Goldberg  is  a noted  authority 
in  virtual  machine  concepts  and  participated  in  the  design  of  the 
VMM. 

• Larry  Shannon --Honeywell  Information  Systems.  Mr.  Shannon 
is  an  implementor  of  the  VMM. 

• Allan  Levy — a scientist  for  BGS  Systems,  Inc.  He  was  responsible 
for  the  BEST/1  modeling  and  analysis  appearing  in  this  document. 

• Dr.  Jeffrey  Buzen--BGS  Systems,  Inc.  Dr.  Buzen  is  an  industry 
leader  in  techniques  of  performance  analysis  and  modeling. 

• W.  Earl  Boebert- -Systems  and  Research  Center.  He  assisted 
with  the  management  of  the  project  for  a three-month  period. 

• Mr.  Donald  Elefante--RADC.  He  provided  the  baseline  work 
load  software  for  Multics. 
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SECTION  II 


PERFORMANCE  ANALYSIS  TECHNIQUES 


CONFIGURATION 

The  RADC  Virtual  Machine  Monitor  consists  of  a modified  6180  processor, 
IOM,  and  Datanet  355.  The  experiments  were  conducted  using  a memory 
configuration  of  512  K words  and  a single  processor  (Figure  1).  When 
executing  a native  mode  experiment,  an  operating  system  was  allowed  to 
use  256  K words  of  memory.  During  VMM  execution  analysis,  the  VMM 
occupied  256  K and  the  virtual  machine  used  the  second  256  K.  In  this  way 
the  memory  utilization  for  a given  operating  system  was  held  constant  for 
comparison  between  native  and  virtual  mode. 


NATIVE  MODE  VMM  MODE 


Figure  1.  Configuration 
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Dual  operating  system  performance  (GCOS/GCOS,  Multics/GCOS,  and 
Multics/Multics)  under  the  VMM  was  not  monitored  due  to  the  time  con- 
straints of  the  project. 

HARDWARE  MODIFICATIONS  FOR  VMM 

The  hardware  support  for  VMM  exists  in  the  68/80  CPUs,  the  IOM  direct 
channels  connected  to  Datanet  355s,  and,  to  a minor  extent,  in  the  Datanet 
355s.  The  CPU  and  Datanet -related  changes  are  described  below. 

CPU  Modifications 

The  CPU  modifications  are  described  under  two  categories,  which  is  the 
sequence  in  which  they  were  implemented. 

Category  1 Changes -- 

Mode  Switch  Position- -The  processor  mode  switch  shall  have  a position 
added  to  it.  Its  positions  shall  be  6000/ 6100/ VMM  (or,  if  you  wish,  GCOS / 
Multics/ VMM).  The  changes  described  in  the  following  paragraphs  shall 
be  active  only  if  the  manual  mode  switch  is  in  the  VMM  position  unless 
otherwise  stated. 

Real/  Virtual  Modes  and  Indicator- -A  bit  shall  be  added  to  the  Indicator 
Register  (IR)  whose  state  shall  indicate  that  the  processor  is  in  "real"  or 
"virtual"  mode.  This  shall  occupy  bit  32  of  the  IR  and  shall  = 1 when  in 
real  mode  and  0 when  in  virtual  mode.  "Real"  mode  will  always  be  entered 
upon  the  execution  of  a transfer  of  control  instruction  while  servicing  an 
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interrupt  or  fault.  This  will  be  the  only  means  of  entering  "real"  mode. 
Once  in  "real"  mode,  the  execution  of  an  RCU  instruction  with  indicator 
bit  32  = 0 will  be  the  only  instruction  which  can  be  used  to  return  to  "virtual" 
mode.  Thus  this  indicator  shall  not  be  affected  by  either  the  LDI  or  RET 
instructions.  The  state  of  this  indicator  will  only  be  storable  by  the  SCU 
instruction. 

Fault  on  Certain  Instructions — Attempted  execution  of  the  following 
instructions  in  virtual  mode  shall  cause  an  IPR  fault  with  an  illegal  in-slave 
indicator  set  in  the  SCU  fault  conditions: 


RSW 

SMCM 

LCPR 

DIS 

SSCR 

RMCM 

SCPR 

RSCR 

SMIC 

CIOC 

An  IPR  fault  instead  of  a command  fault  will  occur  when  any  of  the  above 
instructions  are  executed  in  a GCOS  III  virtual  machine  in  slave  mode. 

Category  2 Changes-- 

VMB  AR  / VMBND — An  additional  base /bound  register  shall  be  added. 
The  base  shall  be  added  to  the  currently  computed  final  address  when 
executing  in  virtual  mode  and  the  bound  shall  be  used  to  bound  the  resulting 
sum.  The  register  and  adder  shall  be  wide  enough  to  apply  relocation  and 
addressing  of  up  to  16  million  words.  The  granularity  of  the  VMBAR  base 
and  bound  shall  be  32  K (or  preferably  less). 

LVMBAR/SVMBAR  Instructions --These  instructions  must  be  added  to 
load  and  store  the  VMBAR  while  in  real  mode.  Their  execution  in  virtual 
mode  shall  result  in  an  IPR  fault  with  the  illegal  in-slave  indicator  set  in 
the  SCU  fault  conditions. 
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The  Jj  VMBAR  will  have  an  op  code  of  210  and  will  load  the  VMBAR  from 
bits  0 through  17  of  C(Y).  The  SVMBAR  with  an  op  code  of  510  will  store 
the  VMBAR  in  bits  0 through  17  of  C(Y)  and  bits  18  through  35  of  C(Y)  will 
be  unchanged. 

VM  Out-of-Bounds  Fault- -If  a virtual  machine  attempts  to  address 
beyond  the  bound  specified  by  the  VMBAR,  a store  fault  with  the  out-of- 
bounds  indicator  set  in  the  SCU  fault  conditions  shall  result. 


6000/6100  Indicator --Bit  33  of  the  indicators  shall  place  the  processor 
in  either  6000  or  6100  mode.  This  indicator  will  be  meaningful  only  when 
the  processor  is  in  VMM  or  6100  manual  switch  position.  (The  use  of  this 
indicator  in  6100  switch  position  is  required  for  the  encapsulation  of  an 
entire  GCOS  system  under  Multics. ) The  indicator  shall  = 0 when  in  6100 
mode  and  s 1 when  in  6000  mode.  It  will  always  be  stored  by  the  SCU 
instruction.  The  SCU  instruction  is  the  only  instruction  which  will  store 
the  state  of  this  indicator.  The  indicator  shall  be  loadable  only  by  the 
RCU  instruction,  a fault,  an  interrupt,  and  the  TSS  instruction.  This 
indicator  shall  affect  the  operation  of  the  processor  in  the  following  manner: 


6100  Mode--The  processor  shall  execute  as  a standard  68/80 
except  as  modified  a£  described  in  this  document.  This  includes 
the  use  of  indicator  bit  28  to  indicate  the  use  of  the  fease  Address 
Register  for  address  development.  It  shall  be  the  software's 
responsibility  to  ensure  that  this  indicator  is  properly  set. 


6000  Mode- -When  in  absolute  mode,  the  processor  shall  operate 


as  S 6000  processor  except  as  modified  for  the  VMM  as  noted  in 
the  previous  subsection.  Thus,  the  processor  shall  perform 
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addressing  relative  to  the  real  memory  origin  or  the  VMBAR  as 
indicated  by  the  state  of  the  real / virtual  indicator. 

In  append  mode  the  processor  shall  only  be  capable  of  executing 
valid  6000  instructions.  The  processor  shall  perform  address 
development  relative  to  the  origin  of  the  segment  indicated  by  the 
PSR  including  page  relocation  if  the  segment  is  paged.  Addresses 
shall  be  further  relocated  by  the  base  contained  in  the  VMBAR  if 
the  processor  is  in  virtual  mode.. 

In  addition,  the  processor  shall  operate  in  the  following  6000 -like 
manner  in  spite  of  other  modes  and  conditions  of  the  processor 
and  specifications  in  SDWs: 

— Instruction  bit  29  = 1 shall  evoke  only  the  offset  part  (address 
register)  of  pointer  registers. 

— TRO  faults  shall  occur  in  slave  mode,  not  in  master  mode  (even 
if  a slave  is  executing  in  a privileged  segment  or  a master  mode 
program  is  executing  in  an  unprivileged  segment). 

--Instruction  bit  28  = 1 shall  cause  interrupt  inhibition  (even  if 
executing  in  an  unprivileged  segment). 

— The  BAR  shall  be  loadable  in  master  mode  and  shall  offset, 
effective  addresses  in  slave  mode. 

Faults  and  Interrupts -- 

VMM  Switch  Position — When  a fault  or  interrupt  occurs,  the  processor 
will  automatically  enter  Absolute,  Real,  6100,  or  Master  Mode  during  the 
execution  of  the  vector  pair. 


However,  the  corresponding  indicators  will  not  be  affected  unless  a transfer 
of  control  is  executed  in  the  vector  pair.  If  a transfer  is  executed,  the 
indicators  will  be  modified  as  described  above  and  as  affected  by  the  trans- 
fer instruction. 

6100  Switch  Position — A fault  or  interrupt  when  in  6100  position  will 
have  the  same  effect  as  in  VMM  position  with  the  exception  of  the  real 
indicator.  This  indicator  does  not  exist  in  this  position. 

TSS--The  TSS  instruction  shall  set  the  6000/6100  indicator  to  6000 
and  the  master/slave  indicator  to  slave  when  it  is  executed  in  either  VMM 
or  Multics  manual  switch  position. 

ABSA  Instruction --The  processor  shall  execute  the  ABSA  instruction  when 
running  in  the  VMM  switch  position  as  follows: 

• Add  the  contents  of  the  VMBAR  to  the  memory  address  when 
accessing  memory  for  indirect  words,  indirect  pairs,  SDWs,  or 
PTWs.  The  VMBAR  shall  be  added  regardless  of  the  virtual/ real 
indicator  state. 

• Perform  address  development  according  to  the  settings  of  the 
Master/Slave,  Absolute/Append,  and  6000/6100  indicators  when 
the  real/  virtual  indicator  specifies  virtual  mode. 

• Use  the  Zero,  Overflow,  and  Exponent  Underflow  indicators  to 
simulate  the  Master/ Slave,  Absolute/ Append,  and  6000/6100 
indicators,  respectively,  when  the  real/  virtual  indicator  specifies 
real  mode. 
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• The  VMBAR  will  not  be  added  in  the  final  address  development 
in  either  virtual  or  real  mode.  Thus  the  address  returned  by 
the  ABSA  shall  always  be  relative  to  the  VMBAR. 

• When  the  real/ virtual  indicator  denotes  real  mode,  the  negative 
indicator  will  be  used  to  specify  the  type  of  address  returned  by 
the  ABSA  instruction.  When  the  negative  indicator  is  off  (*  0), 
the  24 -bit  absolute  operand  address  will  replace  the  most  signif- 
icant 24  bits  of  the  accumulator,  as  is  normally  the  case  with  the 
ABSA  instruction.  However,  when  the  negative  indicator  is  set 
(=  1),  an  18-bit  effective  address  will  be  stored  in  bits  6 through 
23  of  the  accumulator  while  zeroes  will  be  placed  in  the  remaining 
13  bits.  This  18-bit  address  shall  be  developed  by  performing 
all  address  development  normally  associated  with  the  ABSA 
instruction  with  the  exception  of  the  final  append  cycle. 

During  execution  of  the  ABSA  instruction,  the  current  state  of  the  proces- 
sor registers  will  be  used  for  address  development.  It  is  a software 
responsibility  that  these  be  properly  loaded  before  ABSA  execution.  This 
implies  that  the  ABSA  instruction  must  be  executed  in  Absolute/ Real  mode 
when  simulating  VM  address  development  within  the  VMM. 

Transfer  and  Set  Virtual  Instruction- -This  instruction  when  executed  in 
real  mode  will  transfer  according  to  the  operand  address  and  enter  virtual 
mode  (i.  e. , bit  32  of  the  IR  shall  be  reset  to  0).  When  executed  in  virtual 
master  mode,  the  instruction  will  perform  as  a TRA  instruction.  Execu- 
tion of  the  instruction  in  virtual  slave  mode  will  result  in  an  IPR  fault  with 
the  illegal  in-slave  indicator  set  in  the  SCU  fault  conditions.  This  new 
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Instruction  shall  have  an  op  code  of  715  with  bit  27  equal  to  1.  Currently, 
the  use  of  this  instruction  within  the  VMM  is  not  anticipated;  however,  it 
will  be  used  by  off-line  T&D  for  the  VMM  hardware. 

IOM  and  Datanet  355  Modifications 

Hardware  support  is  needed  to  allow  a VMM  to  provide  communication 
support  to  a virtual  machine  in  such  a way  that  neither  the  operating  system 
in  the  virtual  machine  nor  the  355  software  need  be  changed.  In  addition, 
this  support  should  prevent  the  355  from  accessing  6100  memory  beyond 
the  bounds  of  a "window"  which  is  set  by  the  VMM. 

The  hardware  required  shall  consist  of  additions  to  the  direct  channel 
which  shall  modify  its  operation  roughly  as  follows: 

1.  When  a connect  with  a mask  PCW  is  delivered  to  the  direct  channel 
from  the  6100,  it  shall  retrieve  a base  and  bound  from  the  6100 
memory  and  then  process  the  mask  PCW. 

2.  Whenever  the  355  requests  an  access  to  6100  memory,  the  direct 
channel  shall  relocate  and- bound  check  the  address  to  be  accessed. 

This  change  accomplishes  the  desired  result  for  355s  dedicated  to  virtual 
machines.  The  355,  its  software,  and  the  operating  system  continue  to 
operate  as  before  without  visibility  of  the  direct  channel  relocation  and 
bounding.  The  relocation /bound  value  is  set  by  the  VMM  and  is  unchange- 
able by  either  the  355  or  the  virtual  machine  it  is  serving. 
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Each  IOM  direct  channel  which  interfaces  to  a Datanet  355  is  modified  to 
store  the  base  and  bound  addresses  of  a virtual  machine  and  to  assure  that 
all  data  transfers  on  behalf  of  a virtual  machine  remain  within  these 
address  limits.  In  addition,  an  interface  is  added  between  the  direct 
channel  and  each  Datanet  355  which  allows  the  former  to  deliver  an  "out- 
of-bounds"  fault  to  the  latter. 

GCOS  MONITORING  TOOLS 


Early  in  the  project,  it  had  been  assumed  that  standard  GCOS  measuring 
tools  could  be  used  to  measure  the  performance  of  GCOS  under  VMM.  In 
surveying  available  tools,  one  called  Peripheral  Resource  Monitor  (PRM) 
was  thought  to  be  the  best  of  those  available,  particularly  in  regard  to 
graphical  displays.  The  key  measurements  centered  around  processor 
time  both  for  GCOS  and  VMM.  After  some  investigation,  the  "virtual  time 
slippage"  problem  was  investigated  in  some  detail.  The  essence  of  the 
problem  is  as  follows.  In  GCOS  all  time  is  derived  from  the  processor 
interval  timers.  These  are  saved  and  restored  by  VMM  each  time  control 
of  the  processor  is  taken  from  and  returned  to  GCOS.  As  a result,  GCOS 
maintains  accurate  virtual  processor  time,  but  its  real  wall  clock  time  will 
become  inaccurate  when  running  under  VMM.  The  effect  of  "virtual  time 
slippage"  on  a program  like  PRM  (which  measures  processor  idle  directly 
and  processor  busy  by  subtracting  idle  from  elapsed  wall  time)  is  that  the 
processor  is  measured  under  VMM  to  be  100  percent  busy  whether  it  is 
10  percent  busy  or  100  percent  busy. 
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In  an  effort  to  correctly  capture  GCOS  virtual  processor  time  and  also 
provide  such  VMM  data  as  VMM  processor  and  idle  time,  a special  monitor 
program  was  written  called  VMMON.  The  key  design  feature  of  the  program 
was  that  VMMON  read  the  system  controller  clock  which  is  accurate  inde- 
pendent of  software.  This  real  elapsed  time  was  compared  with  GCOS 
time  every  15  seconds  which  under  VMM  showed  a distinct  and  increasing 
discrepancy  as  time  continued.  This  discrepancy  was  the  "virtual  time 
slippage"  which  is  time  when  GCOS  is  not  in  execution,  time  due  to  VMM 
overhead,  and/or  time  when  other  virtual  operating  systems  are  executing. 
Using  this  technique,  VMMON  can  accurately  measure  virtual  GCOS  pro- 
cessor and  idle  time  and,  also,  VMM  processor  overhead  in  an  environ- 
ment which  is  relatively  processor  bound  with  only  one  GCOS  under  VMM. 
The  various  tests  have  proved  out  this  technique  for  the  environment  as 
described. 

In  an  attempt  to  measure  processor  utilization  for  an  environment  of  VMM, 
GCOS,  and  Multics,  the  following  steps  were  taken.  Changes  to  VMM 
were  designed  which  would  record  in  GCOS  memory:  VMM  processor  time, 
real  system  idle  time,  and  Multics  processor  time.  VMMON  was  designed 
to  read  this  data  from  GCOS  memory  and  display  how  this  data  changed  at 
each  sample  period.  Unfortunately,  the  required  patches  to  VMM  were 
not  able  to  be  debugged  given  the  time  constraints  of  the  project. 

A copy  of  the  VMMON  program  is  included  in  Appendix  B. 
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GCOS  WORK  LOAD 


Two  fabricated  work  loads  were  generated  for  the  GCOS  VMM  benchmark 
tests  and  were  designed  to  produce  an  average-type  load  and  an  I/O  bound 
load.  These  loads  tested  the  VMM  performance  under  a range  of  work  load 
types  which  represent  those  typical  of  a data  processing  site. 

In  general  under  GCOS,  the  amount  of  system  processor  overhead  (which 
is  not  charged  to  a user  program)  increases  as  the  level  of  I/O  activity 
increases.  Since  VMM  intercepts  each  I/O  request  and  performs  some 
processing  for  it,  the  VMM  system  processor  overhead  would  also  increase 
as  the  level  of  I/O  activity  increases.  The  fabricated  work  loads  were  also 
designed  to  be  a measurement  of  this  relationship. 

The  average-type  load  was  composed  of  four  GCOS  jobs.  All  jobs  had  the 
same  structure  with  different  I/O  and  processor  parameters.  Each  job 
was  composed  of  three  activities:  activity  1 was  a compilation  of  the 
Fortran  driver  program  with  the  parameters;  activity  2 was  an  assembly 
language  (GMAP)  compilation  of  a subroutine  called  by  the  Fortran  driver 
program  which  was  capable  of  performing  I/O  accesses  at  a maximum  rate; 
activity  3 was  the  execution  of  the  programs  described  in  activities  1 and  2 
which  actually  produced  the  desired  work  load.  The  I/O  to  processor  ratios 
produced  by  the  four  jobs  (by  varying  the  parameters)  were  as  follows: 
job  AV001  = 0.25:1;  job  AV002  = 0.50:1;  job  AV003  = 1:1;  job  AV004  = 2:1. 
The  overall  I/O-processor  ratio  for  the  average  load  was  0.  9:1. 


The  I/O  or  channel  bound  load  was  also  composed  of  four  GCOS  jobs.  All 
jobs  were  identical  except  for  the  job  control  language  which  varied  the 
disk  device  to  which  the  I/O  accesses  were  directed.  This  was  an  attempt 
to  maximize  I/O  accesses  while  minimizing  device  contention.  The 
structure  of  the  job  was  identical  to  that  of  the  average -type  load.  The 
parameters  in  the  Fortran  driver  program  produced  an  I/O-to-processor 
ratio  of  12:1  (which  is  very  I/O  bound).  The  four  job  names  were  CHOU. 
CH012,  CH021.  CH022. 

A copy  of  the  fabricated  programs  described  above  is  included  in  Appendix 
A. 

MULTICS  MONITORING  TOOLS 

The  principal  tool  used  to  monitor  the  performance  of  Multics  was  the 
PL/1  program  termination_overseer.  Other  monitoring  commands  such 
as  total_time_meters  and  page_multilevel_meters  were  used  briefly. 

These  last  two  system  commands  substantiated  the  increase  in  page  fault 
time  as  shown  in  the  BEST/1  analysis  but  proved  to  be  inconclusive  in  the 
absolute  cases. 

The  termination  overseer  program  uses  the  real  time  consumed  by  the 
execution  of  the  job  scripts  to  compute  a number  called  throughput. 
Throughput  is  defined  as  the  number  of  iterations  of  a given  job  divided 
by  total  time  consumed  for  the  job.  These  numbers  for  each  job  script 
are  then  averaged  to  give  the  total  experiment  throughput  under  varying 
loads. 


MULTICS  WORK  LOAD 


The  means  by  which  the  Multics  operating  system  is  driven  in  order  to 
collect  performance  data  is  somewhat  different  than  that  of  GCOS.  Since 
Multics  is  primarily  an  interactive  system,  a simulation  of  interactive 
work  loads  was  used  in  conjunction  with  the  absentee  facility  of  Multics. 

The  programs  used  to  create  the  load  and  in  some  cases  to  collect  the  data 
were  produced  by  RADC  for  monitoring  the  performance  of  new  software 
releases.  However,  they  have  been  modified  in  several  cases  to  fit  the 
needs  of  the  VMM  performance  analysis. 


Absentee  Processing 


The  absentee  facility  of  Multics  is  a means  by  which  jobs  can  be  scheduled 
for  deferred  processing  without  the  need  for  interaction  on  the  part  of  the 
submitter.  These  jobs  retrieve  their  needed  parameters  and  responses 
from  a file  which  has  been  constructed  prior  to  the  execution  of  the  job. 

By  judicious  anticipation  of  the  execution  of  a program,  responses  can  be 
stored  in  Multics  segments  which  cause  a program  to  behave  in  the  desired 
manner.  The  full  complement  of  Multics  commands  is  available  to  an 
absentee  process.  There  are  no  distinctions  between  interactive  processes 
and  these  batch-like  absentee  processes  except  that  input  is  from  segments 
rather  than  from  a terminal.  In  fact,  absentee  processes  may  be  consid- 
ered to  be  interactive  processes  which  are  not  attached  to  a terminal. 
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Job  Sequence 


The  programs  described  here  can  be  found  in  the  appendixes:  Job  Scripts 
and  Monitoring  Tools. 

An  exec_com  or  command  line  file  (ec)  called  "setup"  is  first  executed  to 
determine  the  desired  load  factors  which  control  the  experiment.  Scheduling 
parameters  at  this  phase  include  the  start  time  of  the  experiment,  the 
number  of  processes,  and  the  number  of  interactions  desired  for  each 
process  which  will  be  scheduled.  Once  this  has  been  determined,  the 
requested  number  of  processes  are  entered  into  the  absentee  queue  and 
scheduled  for  execution  at  the  desired  time.  The  final  function  of  setup 
is  to  call  into  execution  a monitoring  program  called  termination  overseer. 
This  monitor  will  be  explained  in  the  section  on  Multics  Monitoring  Tools. 

The  job  which  has  been  scheduled  for  execution  is  named  load  overseer  n 
where  n is  an  integer  tag  indicating  the  number  of  the  particular  process. 

For  example,  if  three  processes  are  desired  to  be  executed  simultaneously, 
then  these  absentee  jobs  queued  are  named  load_overseer_l,  load  overseer  2, 
and  load_overseer_3,  respectively.  The  load_overseer  prototype  consists 
of  a second  exec__com  which  establishes  the  proper  working  directory  for 
the  experiment  and  executes  a driver  program  to  control  the  actual  load 
of  a particular  process. 

The  driver  program  is  called  load_control.  This  program  creates  and 
initializes  the  necessary  files  for  collecting  the  data  during  a single  exper- 
iment. The  clock  is  read  before  and  after  each  iteration  of  an  internal 
loop  and  the  real  time  used  is  accumulated  in  a data  segment.  The  internal 
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loop  is  controlled  by  the  iteration  count  specified  as  an  argument  to  the 
setup  procedure.  At  each  pass  through  the  loop,  a program  to  exercise 
the  system  is  called.  This  exerciser  program  is  called  load.  Load  uses 
a standard  Multics  performance  monitoring  device  called  flush.  Flush 
modifies  each  of  256  pages  in  the  user's  process  space  and  then  compares 
the  results.  In  this  way,  at  least  256  page  faults  occur.  Additionally,  load 
executes  a sequence  of  processor  instructions  designed  \o  cause  processor- 
bound  activity  a variable  number  of  times  (set  to  100  for  these  experiments). 

When  load  terminates  after  causing  the  paging  and  processor  activity, 
return  is  made  to  the  load_control  iteration  loop.  This  process  is  continued 
until  the  desired  iteration  count  for  this  particular  absentee  job  has  been 
exhausted.  Load_control  then  accumulates  and  computes  the  throughput 
factor  for  this  job  and  terminates.  This  concludes  the  life  of  a single 
process. 

Once  all  processes  have  terminated,  the  termination  overseer  computes 
the  total  throughput  for  all  absentee  jobs  and  terminates.  Figure  2 describes 


the  basic  flow  through  the  load  experiment. 
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ANALYSIS  RESULTS 


There  are  demonstrable  penalties  in  running  a job  stream  in  a virtual 
machine  rather  than  in  a real  machine  environment.  Such  penalties  include 
the  consumption  of  extra  system  resources  (e,  g. , processor  cycles)  as 
well  as  potential  degradation  to  various  measures  of  overall  performance 
(e.  g. , response  time  and  throughput).  In  this  section  the  performance 
impact  of  running  the  GCOS  and  Multics  operating  systems  under  the 
Virtual  Machine  Monitor  (VMM)  versus  under  the  native  machine  environ- 
ment is  examined. 

This  section  of  the  report  is  divided  into  two  parts:  the  analysis  of  GCOS 
and  the  analysis  of  Multics.  In  each  part,  the  benchmark  experiments  run 
by  Honeywell  are  discussed  first.  Next,  the  analysis  of  the  benchmark 
data  to  develop  an  analytical  performance  model  is  described.  Finally, 
the  results  of  evaluating  the  performance  model  are  presented. 


BENCHMARK  ANALYSIS 


Honeywell  performed  a variety  of  benchmark  experiments  in  both  a native 
GCOS  environment  (i.  e. , in  a non-virtual  machine  mode)  and  in  a VMM/ 
GCOS  environment  (i.  e. , under  VMM  control).  These  experiments  were 
performed  in  a completely  isolated  environment;  no  system  activity  other 
than  that  defined  in  the  benchmark  was  present.  The  hardware  configuration 
on  which  the  benchmarks  were  run  is  presented  in  Figure  3. 


CM.V  A SINGLE  PROCESSOR  MS  USED  DURING  THE  BENCHMARK. 

Figure  3.  RADC  Multics  (VMM)  Configuration 

Three  test  series  were  run  on  both  native  GCOS  (on  a Multics  system  with 
VMM  processor)  and  GCOS  under  VMM.  The  test  series  spanned  the 
spectrum  of  processor  bound  to  channel  bound,  thus  reflecting  how  VMM 
performs  under  a variety  of  loads.  The  three  separate  benchmarks  con- 
sisted of: 


Compute-Bound  Activity- -heavily  CPU  demanding  tasks. 
Channel-Bound  Activity- -heavily  I/O  demanding  tasks. 
Average-Load  Activity- -average  CPU  and  I/O  demanding  tasks 


ire  monitors  were  used  during  each  benchmark  experiment.  The 
source  Monitor  (SRM),  an  updated  version  of  a previous  monitor 
was  available  from  Honeywell  Information  Systems.  The 


Virtual  Machine  Monitor  (VMMON)  was  developed  specifically  for  the 
benchmark  analysis  effort. 

In  the  native  GCOS  environment,  both  monitors  provided  substantially  the 
same  results.  Under  the  VMM  environment,  however,  SRM  was  completely 
inadequate.  This  was  largely  due  to  virtual  time  slippage.  Basically, 
virtual  time  slippage  refers  to  the  difference  in  time  kept  by  GCOS  and  the 
actual  or  real  time.  The  GCOS  clock,  being  an  internal  software  clock, 
is  updated  by  hardware  timer  interrupts  when  GCOS  is  active.  In  the 
virtual  machine  environment,  during  periods  when  the  VMM  is  active  and 
GCOS  is  idle,  timer  interrupts  intended  to  update  the  GCOS  clock  are  lost 
since  they  are  not  enabled.  This  causes  a discrepancy  between  the  actual 
time  and  the  time  perceived  by  GCOS.  The  SRM,  in  its  attempts  to  obtain 
timing  information  in  the  virtual  machine  environment,  unknowingly  accesses 
GCOS's  software  clock.  VMMON  on  the  other  hand  compensates  for  the 
situation  by  accessing  the  system  controller  clock,  a microsecond  clock 
which  remains  accurate  under  both  the  native/ GCOS  and  the  VMM/ GCOS 
environments.  In  addition,  VMMON  reports  both  the  value  of  the  GCOS 
time  and  the  actual  time,  thereby  permitting  the  virtual  time  slippage  to 
be  deduced. 

In  the  VMM/ GCOS  environment,  the  virtual  time  slippage  is  caused  only 
by  VMM  overhead  activity  and  system  idle  activity.  The  virtual  time 
slippage,  therefore,  provides  a convenient  mechanism  for  establishing 
overhead  attributable  to  the  VMM.  One  need  only  concentrate  on  periods 
in  the  native  system  where  the  processor  is  known  to  be  100  percent  busy. 
Virtual  time  slippage  will,  in  this  case,  accurately  represent  VMM  over- 
head only. 
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Two  of  the  benchmark  test  series  run  in  the  native/ GCOS  environment --the 
processor  bound  and  the  average  load  activities- -managed  to  consume  100 
percent  of  the  processor.  In  the  channel  bound  tests,  however,  the  native/ 
GCOS  was  approximately  66  percent  idle  and  so  the  VMM  overhead  could 
not  be  deduced  without  additional  data.  Attempts  to  obtain  this  data  through 
applications  of  various  patches  to  the  VMM  were  not  successful. 


The  actual  monitor  reported  data  for  all  benchmark  experiments  is  provided 
in  Tables  1 through  4.  This  data  has  been  reduced  for  use  in  the  perfor- 
mance analysis  presented  below.  Table  1 presents  a summary  of  processor 
and  channel  usage  data.  Tables  2 through  4 present  data  extracted  from 
VMMON  for  six  time  intervals  T1  through  T6  in  both  the  native/ GCOS  and 
VMM/ GCOS  environments.  The  following  summarizes  the  information 
presented: 

GCOS  Time:  time  interval  captured  by  GCOS 


Real  Time: 
GCOS  Ovhd: 
VMM  Ovhd: 
Idle: 

Connects: 


time  interval  captured  by  the  system  controller  clock 

percentage  of  GCOS  overhead  time 

percentage  of  VMM  overhead  time 

percentage  of  processor  idle  time 

number  of  connects  per  second  of  GCOS  time 
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*For  all  job  activities 

**The  VMM  GCOS  tests  were  terminated  early  due  to  time  constraints 


Native  GCOS  Environment 


^''•■^.^Interval 

T1 

T2 

T3 

T4 

T5 

T6 

Statistic*" 

GCOS  Time  (sec) 

94.  4 

223.2 

349.5 

128.8 

220.7 

443.0 

Real  Time  (sec) 

94.4 

223.2 

349.4 

128.8 

220.7 

442.  9 

% GCOS  Ovhd 

2.4 

1.  8 

1.7 

1.4 

1.  6 

1.7 

% VMM  Ovhd 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

% Idle 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Number  of 
Connects/ sec 

5.3 

2.  9 

2.5 

1.1 

2.0 

2.4 

VMM/ GCOS  Environment 

t e r va  1 

T1 

T2 

T3 

T4 

T5 

T6 

Statistic  " — 

GCOS  Time  (sec) 

124.  9 

190.  5 

286.0 

287.  5 

415.  9 

479.0 

Real  Time  (sec) 

140.  4 

208.  8 

306.  5 

295.4 

427.2 

501. 9 

% GCOS  Ovhd 

3.  2 

2.7 

2.1 

0.9 

0.9 

1.2 

°Ic  VMM  Ovhd 

12.  4 

9.6 

7.2 

2.7 

2.7 

4,8 

°!c  Idle 

0.  0 

0.0 

0.0 

0.0 

0.0 

0.0 

Number  of 
Connects/  sec 

8.  3 

6.  9 

4.  8 

1.2 

1.  1 

n 

TABLE  3.  SUMMARY  OF  RESULTS  FOR  AVERAGE  WORK  LOAD 
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Native  GCOS  Environment 


— ^^Interval 

Statistic 

T1 

T2 

T3 

T4 

T5 

T6 

GCOS  Time  (sec) 

124.2 

216.5 

339.  6 

274.5 

216.  1 

340.  1 

Real  Time  (sec) 

124.  1 

216.4 

339.  4 

274.3 

216.0 

339.9 

% GCOS  Ovhd 

10.2 

10.0 

10.0 

9.5 

8.7 

9.8 

% VMM  Ovhd 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

% Idle 

0. 16 

0.  14 

0.09 

0.09 

0.  12 

0.0 

Number  of 
Connects/sec 

38.3 

38.0 

38.0 

35.  9 

32.  3 

37.4 

VMM/GCOS  Environment 


^^■^.^Interval 

Statistic 

T1 

T2 

T3 

T4 

T5 

T6 

GCOS  Time  (sec) 

154. 1 

278.4 

186.  1 

306.  9 

400.2 

Real  Time  (sec) 

184.  8 

334.  1 

548.  9 

223.3 

364.  1 

479.8 

% GCOS  Ovhd 

9.  9 

9.9 

9.3 

9.9 

9.  1 

9.1 

% VMM  Ovhd 

19.9 

20.0 

19.  1 

20.0 

18.6 

19.9 

% Idle 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Number  of 
Connects/ sec 

36.7 

36.8 

34.7 

36.  9 

33.7 

36.6 

TABLE  4.  SUMMARY  OF  RESULTS  FOR  CHANNEL-BOUND 
WORK  LOAD* 


Native/  GCOS  Environment 


Interval 

Statistic 

T1 

T2 

T3 

T4 

T5 

T6 

GCOS  Time  (sec) 

142.  9 

264.0 

324.0 

181.0 

271.2 

479.  2 

Real  Time  (sec) 

142.  8 

263.  9 

323.  9 

181.0 

271.  2 

479.2 

°7c  GCOS  Ovhd 

17.4 

17.6 

17.  6 

17.7 

17.  6 

17.  6 

% VMM  Ovhd 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

°!c  Idle 

65.8 

65.  8 

65.  9 

65.  9 

66.  1 

66.2 

Number  of 

Connects/ sec 

64.  0 

64.  9 

64.  9 

65.  5 

65.2 

65.0 

*Since  native  GCOS  idle  is  significantly  greater  than  zero  (approximately 
66  percent),  VMM  overhead  for  the  channel-bound  work  load  cannot  be 
deduced.  Attempts  to  patch  VMMON  to  present  the  data  on  the  VMM/ 
GCOS  environment  were  unsuccessful. 
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GCOS-VMM  OVERHEAD  REGRESSION  ANALYSIS 


The  quantity  of  interest  for  the  analyses  that  follow  is  the  VMM  overhead. 
This  quantity  represents  the  processor  time  consumed  by  the  VMM  in 
servicing  users'  requests  for  system  resources.  The  approach  to  be 
used  to  compute  VMM  overhead  is  described  below. 

The  various  time  periods  for  which  measurements  are  available  are  de- 

1 noted  by  T,,  (i  = 1,2, ... , n).  The  measured  VMM  overhead  time  (i.  e. , 

the  virtual  time  slippage  in  the  ith  time  period)  is  denoted  by  L.  Suppose 


29 


there  are  M different  types  of  requests  which  the  VMM  must  service.  If 

iL 

n„  (j-1, . . .m)  is  the  measured  quantity  of  requests  of  type  j during  the  i 

interval  and  0.  is  the  VMM  overhead  incurred  in  servicing  the  type  j 

J th 

request,  then  the  total  VMM  overhead  time  spent  in  the  i time  interval 

may  be  approximated  by: 

t.  = 

1 


m 


z 

3=1 


0.  N.. 
3 13 


(1) 


The  coefficients  0,  , 0_, ....  0 can  be  determined  by  the  method  of  least 
L z m 

squares  so  as  to  minimize  the  sum  of  the  squares  of  the  residuals,  i.  e. , 

n 

mins(ve2 v = Zri 

i=l 

where  residual  r is  defined  as 


m 


r. 

l 


t.  - V 0.  N.. 

i ^ 3 13 

i=l 


If  the  r.  turn  out  to  be  relatively  small,  then  Equation  (1)  is  considered  to 
yield  a suitable  fit  to  the  data.  The  goodness  of  fit  criteria  incorporated 
below  is  the  multiple  correlation  coefficient  obtained  by  comparing  the 
sum  of  the  residual  squares  to  the  sum  of  the  squares  of  the  deviation  of 
the  measurements  from  their  mean  value  (t),  i.  e. , 


1 


Based  upon  the  information  obtained  from  the  benchmark  analyses,  two 

resource  quantities  were  selected  as  being  major  contributions  to  VMM 

overhead.  For  the  ith  time  interval,  T.,  these  were 

1 

• n.^--the  CPU  busy  time  during  T. 

• n^2~-the  number  of  channel  connects  requested  during  T. 
Equation  (1)  in  this  case  reduces  to 


Using  the  benchmark  experiment  data  in  Tables  1 through  4 (excluding  the  ' 
channel-bound  work  load  for  reasons  previously  explained),  the  method  of 
least  squares  was  applied  to  the  equations  represented  in  Table  5. 


TABLE  5.  VMM  OVERHEAD  REGRESSION  DATA 


Work  I xjad 

8 

VMM  Overhead 
Time  (t.)  in  T. 
(sec) 

if 

Total  Number  of 
Connects  (n^)  in  T^ 

i 

15.  5 

124.9 

1042.  9 

KH 

18.  3 

190.  5 

1316.4 

Processor 

20.  5 

286.0 

1381.4 

Bound 

m 

7.  9 

287.  5 

345.0 

5 

11.3 

415.  9 

474.  1 

6 

22.  9 

479.0 

1005.  9 

1 

30.7 

154.  1 

5661.6 

2 

55.  7 

278.  4 

10242.3 

Average 

3 

87.9 

461.0 

16010. 5 

Load 

4 

37.2 

186.  1 

6876.4 

5 

57.2 

306.  9 

10351.7 

6 

79.  6 

400.  2 

14647.3 

The  results  of  the  regression  analysis  of  the  data  in  Table  5 yielded  the 
following: 


el 

°2 

R2 

0.0351 

0.0045 

0.98 

These  results  indicate  that  the  processor  overhead  incurred  through  usage 
of  the  VMM  is  approximately  0.0045  seconds  overhead  for  each  channel 
connect  and  an  additional  3.5  percent  of  non-idle  prooeasoi'  time.  The 
extremely  high  correlation  coefficient  indicates  that  these  results  repre- 
sented a suitable  fit  to  the  data. 


It  should  be  pointed  out  that,  although  an  extremely  high  correlation  coeffic- 
ient has  been  achieved,  the  results  may  not  be  statistically  precise.  The 
method  on  which  the  coefficient  values  are  based  depends  on  the  assumption 
that  the  coefficients  are  constant--not  variable.  This  assumption  is  not 
valid  for  the  data  in  Table  5.  Clearly  different  user  requests  place  signif- 
icantly different  demands  on  the  system,  for  example.  The  true  accuracy 
of  these  coefficients,  therefore,  has  not  been  established.  It  is  possible, 
however,  with  additional  measurements,  to  construct  more  statistically 
accurate  results.  This  remains  a source  for  future  investigations. 

I fl 

Another  point  is  also  of  interest  at  this  time.  Much  care  was  taken  in 
obtaining  representative  time  intervals  including  the  effect  of  transitory 
periods  of  system  activity  rather  than  just  intervals  of  pure  work  load 
activity.  This  was  done  to  include  the  effects  of  transitory  system  activity 
in  the  analyses  of  subsequent  sections.  With  careful  selection  of  periods 

I 
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of  measurement,  it  is  possible  to  distinguish  between  periods  of  pure  work 

load  activity  and  periods  of  combined  work  load  and  transitory  system 

activity.  It  may  be  of  interest  to  determine  the  coefficients  0'  and  0'  in 

1 2 

this  case.  The  values  of  0.  would  more  accurately  isolate  the  contributions 
of  n.j  and  n.2  to  the  VMM  overhead. 


For  this  purpose  the  following  intervals  were  chosen  as  representative  of 
pure  work  load  activity  (i.  e. , little  or  no  system  transitory  effects  were 
included): 


Work  Load 

VMM  Overhead 
(t.)  (sec) 

GCOS  Busy 
Time  (njj) 
(sec) 

Number  of 
Connects 
(nil) 

Processor 

Bound 

6.8 

287.3 

132.9 

Average 

Load 

55.7 

278.4 

10242.3 

Repeating  the  previously  described  regression  analysis  determines: 


Oj'  = 0.0215 
02'  = 0.0048 

Application  of  these  new  coefficients  to  intervals  which  include  transitory 
effects  yields  a significant  difference  between  the  observed  and  predicted 
VMM  overhead.  This  effect  is  due  to  overhead  resulting  from  factors  other 
than  the  GCOS  busy  time  (n.j)  and  the  number  of  connects  (n^),  that  is. 
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ni3#  n.4# . . . which  have  not  been  identified.  To  determine  such  factors 
would  require  substantial  additional  data  and  analyses. 

BEST/ 1 -GCOS  ANALYSIS 
tm 

Based  on  the  values  of  and  ©2  derived  in  the  pi  evious  section,  it  was 
possible  to  hypothesize  several  configuration  and  work  load  alternatives 
and  determine  their  respective  performance  degradation  when  executing 
GCOS  in  a virtual  machine  rather  than  in  a native  machine  environment. 
This  was  done  with  the  help  of  BGS  Systems'  proprietary  modeling  package 
BEST/1^.  First,  the  performance  of  each  of  the  hypothesized  alterna- 
tives running  in  a native/ GCOS  environment  was  analyzed  using  BEST/1.  . 

tm 

Next  the  previously  determined  coefficients,  and  0 2,  were  used  to 
determine  the  VMM  overhead  degradation  of  the  hypothesized  work  loads 
in  the  VMM/ GCOS  environment.  Finally,  BEST/ltm  was  again  used  to 
determine  the  performance  impact  of  executing  that  same  configuration 
and  work  load,  under  GCOS,  in  the  virtual  machine  environment. 

The  configuration  and  work  load  models  constructed  for  the  BEST/  ltm 
analysis  consisted  of  a canonical  job  stream  executing  on  a single  proces- 
sor system,  in  one  case  including  and  in  one  case  excluding  the  effects 
of  I/O  device  contention.  Individual  tasks  in  the  job  stream  were  assumed 
to  consume  approximately  1 second  of  CPU  time  and  to  perform  a varying 
number  of  I/O  operations  (0  to  50)  at  approximately  35  M-sec  per  connect. 
The  analyses  were  performed  for  the  job  stream  under  several  distinct 
levels  of  system  load,  ranging  from  a single  thread  environment  to  a level 
of  threading  equal  to  15. 
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The  models  chosen  and  analyzed  via  BEST/ltm  were  directed  toward  deter- 
mining the  VMM  overhead  impact  on  two  important  measures  of  system 
performance:  response  time  and  system  throughput.  Sample  BEST/1 

tm 

modeling  details  and  the  results  of  the  analyses  are  presented  in  Figures  4 
through  12. 
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Figure  4.  Sample  BEST/ltm  Internal  Model  and  Principal 
Results  Report 
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VMM/GCOS 


HEAVY  LOAD 
(MPL  = 15) 


NATIVE/GCOS 
VMM/GCOS 


MEDIUM  LOAD 
(MPL  = 9) 


NATIVE/GCOS 
VMM/GCOS  VMM/GC0S  NATIVE/GCOS 


LIGHT  LOAD 
(MPL  = 3) 
SINGLE  THREAD 
(MPL  * 1) 


2 10  25  50 

NUMBER  OF  CONNECTS/SECOND  OF  PROCESSOR  BUSY 


Figure  7.  Effect  of  VMM  vs.  Native  GCOS  on  Response  Time  as  a 
Function  of  I/O  Activity  (I/O  Contention  Excluded) 


THROUGHPUT 

(UOBS/HOUR) 


HUMBER  OF  CONNECTS/SECOND  OF  PROCESSOR  BUSY 


Figure  9.  Effect  of  VMM  vs.  Native  GCOS  on  Throughput  as  a 
Function  of  I/O  Activity  (I/O  Contention  Included)  • 
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Figure  11.  Effect  of  VMM  vs.  Native  GCOS  on  Throughput  as  a 
Function  of  I/O  Activity  (I/O  Contention  Excluded) 
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MULTICS  BENCHMARKS  AND  OVERHEAD  ANALYSIS 


The  benchmark  experiments  run  by  Honeywell  for  Multics  differ  signifi- 
cantly from  their  GCOS  counterparts.  Rather  than  defining  several  distinct 
work  load  types  and  performing  a series  of  benchmarks  for  each,  as  was 
done  with  GCOS,  only  a single  work  load  type  was  defined  for  the  Multics 
case.  This  work  load  consisted  of  a single  PL/ 1 program  which  itself 
issued  primarily  processor  consuming  requests  through  a large  iterative 
loop  (the  effect  of  flush  was  also  considered) * The  work  load  was  run  in 
both  the  native/ Multics  and  VMM/ Multics  environments  with  three  distinct 
levels  of  threading  (multiprogramming  levels  1,  15,  and  20). 

Statistics  were  gathered  for  this  benchmark  series  through  the  use  of  the 
Multics  System  Metering  tools.  This  tool,  however,  was  not  applied 
solely  to  the  intervals  of  interest  (i.  e. , the  intervals  in  which  the  PL/ 1 
work  load  was  active).  Therefore,  the  data  that  was  collected  and  reported 
turned  out  to  be  insignificant. 

Fortunately,  another  source  of  information  was  available:  a PL/ 1 source 
program  referred  to  as  LOAD -CONTROL.  This  program  was  used  to 
monitor  the  work  load  and  report  the  number  of  iterations  per  minute  that 
it  achieved.  This  provided  a single  data  point  in  each  environment  (Table  6) 
which  was  somewhat  useful  in  determining  the  VMM  overhead  degradation. 

There  are  several  factors  which  affected  the  usefulness  of  the  LOAD- 
CONTROL  data.  Foremost  was  the  fact  that  configurations  for  the  native/ 
Multics  and  VMM/ Multics  were  not  identical.  The  PL/1  work  load  was 
run  under  native/ Multics  without  the  use  of  a high-speed  cache  memory 
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and  under  VMM/Multics  with  use  of  the  cache.  Since  the  cache  memory 
has  a significant  effect  on  such  factors  as  instruction  fetch  rates,  perfor- 
mance analyses  using  this  data  would  clearly  be  affected. 

TABLE  6.  BENCHMARK  DATA  FROM  THE  PL/ 1 
LOAD-CONTROL  PROGRAM 


♦With  high-speed  memory  cache. 
♦♦Without  high-speed  memory  cache. 


... 

. 

Even  so,  a first-order  analysis  of  the  data  was  attempted.  Based  on  the 
above  data,  it  was  deduced  that  under  the  native/ Multics  environment  16.76 
seconds  were  required  per  iteration  while  under  VMM/Multics  this  number 
increased  to  19.05  seconds  per  iteration;  this  implied  an  observed  VMM 
overhead  degradation  of  13.7  percent.* 


♦This  represents  an  increase  factor  of  4 over  the  degradation  for  the  proces- 
sor busy  contribution  in  the  GCOS  use.  These  numbers  are  not  intended  to 
be  compared,  however.  Paging  activity,  for  example,  which  is  not  present 
in  the  GCOS  environment,  causes  a significant  performance  impact  in  the 
Multics  environment.  The  effect  of  Multics  paging  activity  has  in  effect  been 
aggregated  into  the  Multics  processor  busy  contribution.  Explicitly  deter- 
mining the  degradation  contributed  by  paging  would  require  additional  bench- 
mark experiments  and  substantial  further  investigation. 


45 


Since  in  the  benchmark  experiment  for  the  VMM/Multics  case  the  cache 
memory  was  incorporated,  this  figure  represents  a lower  bound  on  the 
VMM  overhead  degradation. 

The  actual  degradation  would  clearly  be  more  significant  for  larger  effective 
iteration  speedups  contributed  by  use  of  the  cache.  Since  typical  speedup 
factors  introduced  by  the  use  of  a high-speed  cache  may  range  from  5 to  30 
percent  for  processor  bound  work  loads,  this  leads  to  the  ranges  of  VMM 
overhead  degradation  depicted  in  Table  7. 

TABLE  7.  ESTIMATED  VMM  OVERHEAD  DEGRADATION 
AS  A FUNCTION  OF  CACHE  CONTRIBUTION 


Effective  Cache 
Contribution  (%) 

Estimated  VMM/Multics 
(Second/ iteration  without  cache) 

Percent  Degradation 
VMM:  Native 

0 

19.05 

13.7 

5 

20.05 

19.6 

10 

21. 17 

26.3 

15 

22.41 

33.7 

20 

23.81 

42. 1 

25 

25.4 

51.6 

30 

27.2 

62.3 
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BEST/r  -MU LTICS  ANALYSIS 
tm 

Even  with  the  minimal  amount  of  data  provided  by  the  benchmark  experi- 
ments in  the  Multics  case,  examination  of  the  performance  impact  of  the 
VMM  degradation  for  a variety  of  hypothesized  configuration  and  work  load 
alternatives  was  attempted.  This  was  done  with  the  aid  of  BGS  Systems' 
proprietary  modeling  package  BEST/ltm. 

For  purposes  of  this  analysis,  the  ©2  factor  in  the  VMM/ Multics  case  was 

assumed  to  be  the  same  as  that  in  the  VMM/GCOS  case  (0.0045  seconds/ 

connect).  The  0^  factor  was  varied  over  the  end  points  of  the  range  of 

effective  cache  contributions  described  previously.  The  performance  of 

the  hypothesized  systems  in  the  native  machine  mode  under  Multics  was 

analyzed  using  BEST/1^.  Next  the  coefficients  0^  and  ©2  were  used  to 

determine  the  VMM  overhead  degradation  of  the  hypothesized  systems 

under  Multics  in  the  virtual  machine  mode.  BEST/1  was  then  used 

tm 

again  to  determine  the  performance  impact  of  executing  the  hypothesized 
systems  under  VMM/ Multics. 

As  in  the  BEST/ 1^-GCOS  analysis,  the  configuration  and  work  load 
models  consisted  of  a canonical  job  stream  executing  on  a single  processor 
system.  Two  hardware  configurations  were  modeled:  one  including  the 
effects  of  I/O  device  contention,  and  one  excluding  these  effects.  Individual 
tasks  in  the  job  stream  were  assumed  to  consume  approximately  1 second 
of  processor  time  and  to  perform  a varying  number  of  I/O  operations  (0  to  50) 
consuming  approximately  35  msec  per  connect.  The  analyses  were  performed 
for  the  job  stream  under  several  distinct  levels  of  system  load,  ranging  from 
a single  load  to  a load  level  of  15. 
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The  models  chosen  and  analyzed  via  BEST/l^m  were  directed  toward  deter- 
mining the  VMM  overhead  impact  on  two  important  measures  of  system 
performance:  response  time  and  system  throughput.  The  results  of  the 
analysis  are  presented  in  Figures  13  through  16. 

SUMMARY  OF  RESULTS 


Initial  benchmark  experiments  and  measurement  data,  obtained  for  the 
Honeywell  6180  configuration  at  RADC,  have  provided  an  indication  of  the 
performance  of  executing  job  streams  under  the  GCOS  and  Multics  oper- 
ating systems  in  both  native  and  virtual  machine  environments.  Analysis 
of  this  data  through  multiple  linear  regression  and  queing  network  tech- 
niques provided  insight  into  the  performance  degradation  of  running  work 
loads  under  GCOS  and  Multics  in  both  the  native  and  virtual  machine  modes. 
The  following  was  observed: 

• For  both  GCOS  and  Multics,  both  measures  of  system  performance- - 
response  time  and  throughput --showed  that  VMM  overhead  had  its 
greatest  effect  on  work  loads  exhibiting  an  intermediate  amount  of 
I/O  activity  (15  to  35  connects  per  second  of  processor  busy  time). 

• In  case  of  GCOS,  the  performance  impact  of  the  VMM  overhead 
increases  with  the  load  for  processor  bounded  work  loads  and 
decreases  with  the  load  for  I/O  bounded  work  loads.  For  work 
loads  exhibiting  a small  amount  of  I/O  activity,  VMM  overhead 
has  only  a minor  effect  on  performance.  For  work  loads  exhibiting 
a large  amount  of  I/O  activity,  contention  at  the  I/O  devices  be- 
comes the  limiting  factor  and  again  the  VMM  overhead  contributes 
only  a minor  effect.  If  contention  at  the  I/O  devices  could  be 
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NUMBER  OF  CONNECTS/SECOND  OF  PROCESSOR  BUSY 


Figure  13.  Ratio  of  VMM:  Native  Response  Time  for  Multics  as 
a Function  of  I/O  Activity  (I/O  Contention  Included) 


NUMBER  OF  CONNECTS/SECOND  OF  PROCESSOR  BUSY 


Figure  15.  Ratio  of  VMM:  Native  Throughput  for  Multics  as  a 
Function  of  I/O  Activity  (I/O  Contention  Included) 
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NUMBER  OF  CONNECTS/SECOND  OF  PROCESSOR  BUSY 

Figure  16.  Ratio  of  VMM:  Native  Throughput  for  Multics  as  a 
Function  of  I/O  Activity  (I/O  Contention  Excluded) 


eliminated  in  the  latter  case,  however,  the  VMM  overhead  would 
again  become  a major  source  of  performance  degradation.  Similar 
statements,  although  probably  applicable  to  Multics,  are  not  con- 
clusive because  of  insufficient  benchmark  data  in  the  Multics  case. 

• In  GOOS  the  VMM  overhead  was  determined  to  be  in  the  range  of 
15  to  28  percent  depending  upon  the  degree  to  which  the  job  mix 
was  I/O  dominated.  The  refinement  of  these  factors  in  terms  of 
processor  busy  time  and  the  I/O  activity  is  3. 5 percent  and  4.  5 
msec  of  overhead  per  connect,  respectively. 

• In  Multics  a substantially  higher  value  of  VMM  overhead  was 
determined.  The  range  of  this  value  was  estimated  to  be  very 
wide,  13  to  60  percent,  because  of  the  insufficient  number  of 
benchmark  experiments.  Overhead  of  greater  than  30  percent 
is  likely. 

• The  relatively  higher  values  for  the  Multics /VMM  degradation  vs. 
GCOS/VMM  degradation  are  attributed  to  the  higher  VMM  over- 
head required  to  process  page  faults  and  associated  I/O  within 
the  Multics  virtual  machine. 

• Reference  to  the  Gj  (A  = 2.  2 percent)  and  02  (B  = 4.  8 msec/ con) 
values  for  the  "pure"  work  loads  reflects  minimum  VMM  overhead 
when  GOOS  is  not  issuing  master  mode  instructions. 
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SECTION  IV 


EVOLUTION  OF  THE  VMM 


VMM  APPLICATIONS 


Early  experience  with  the  VMM  has  shown  that  the  concept  has  a major 
impact  on  the  development  of  operating  system  software.  More  recently, 
potential  assistance  to  the  application  programmer  has  come  to  light  as  a 
result  of  VMM  research.  The  Honeywell  VMM  provides  a minimum  set 
of  capabilities  which  allow  RADC  to  evaluate  this  concept  in  a limited 
environment.  Some  discussion  of  the  major  applications  will  help  focus 
this  evaluation. 


Virtual  Microcomputers  and  Minicomputers 

It  is  convenient  to  describe  the  many  levels  of  processing  (mini,  micro, 
macro,  etc. ) and  the  units  which  they  use  as  a computational  theater. 

Such  a theater  is  a formally  described  processing  system  including  a basic 
idea  of  computation  which  is  independent  of  level.  It  is  possible  for  a VMM 
to  provide  a theater  which  can  produce  multiple  copies  of  itself.  Within 
each  theater,  then,  a particular  instance  of  a processing  unit  could  be 
created; 


Since  VMMs  provide  complete  hardware/ software  interfaces,  multiple 
simultaneous  users,  and  fictitious  I/O  devices,  the  advantages  of  large 
machines  can  be  extended  downward  fof  small  machines.  A large  number 
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of  mini  or  microcomputers  could  be  encapsulated  on  a single  large  system 
executing  a VMM.  This  would  allow  extension  of  the  capabilities  of  small 
machines  for  software  development,  multiprogramming,  and  computer 
systems  research. 

Networking 

Given  a situation  where  the  software  for  a geographically  distributed  system 
needs  to  be  developed  but  only  a single  hardware  system  is  available,  a 
VMM  can  provide  the  needed  test  bed.  By  establishing  multiple  VMs  within 
the  VMM,  each  virtual  machine  can  communicate  with  the  outside  world  and 
other  machines  on  the  pseudo-network  without  actual  communication  lines. 
The  information  can  move  from  one  VM  to  another  VM  by  passing  through 
the  VMM  or  by  going  outside  the  machine  to  a wrap-around  communications 
device. 

Program  Debugging 

One  possible  way  to  use  the  VMM  approach  for  debugging  software  is  to 
allow  a virtual  machine  to  be  "smart"  about  its  environment,  that  is,  allow 
a VM  to  understand  its  interface  with  the  VMM  and  to  know  that  other  VMs 
exist  on  the  same  host  hardware.  In  this  way  a "spy"  program  can  be 
executed  on  VM1  which  views  the  execution  of  VM2,  traps  data  about  that 
execution,  and  provides  a debug  interface  to  a user  of  VM1.  The  spy 
concept  was  implemented  at  the  IBM  Grenoble  Scientific  Center  on  a modi- 
fied CP- 67. 
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Other  software  debugging  aids  using  VMM  concepts  are  extended  console 
functions  and  recursion.  With  a virtual  operator's  console  simulated  on  a 
user's  terminal,  many  system  commands  can  be  extended  out  to  the  user. 
Examination  and  modification  of  absolute  addresses  and  dynamic  modifica- 
tion of  system  scheduling  parameters  are  two  of  these.  The  aspect  of 
recursion  can  be  called  nested  VMMs.  In  this  use  of  a VMM,  a specialized 
debug-oriented  VMM  is  executed  under  the  bare  machine  monitor.  Thus  a 
VMM  becomes  a VM.  Obviously,  efficiency  considerations  may  limit  the 
depth  to  which  recursion  is  feasible. 

Input /Output  Applications 

Two  major  applications  of  VMM  are  I/O  program  analysis  (virtual  I/O) 
and  new  peripheral  support.  The  first  of  these  was  discussed  in  the  interim 
report  and  will  not  be  elaborated  upon  here.  The  extension  of  the  virtual 
I/O  concept  allows  us  to  visualize  three  scenarios  in  which  a VMM  is  used 
to  aid  the  introduction  of  a new  peripheral. 


Scenario  1.  A New  Peripheral  Device  is  Being  Proposed  or  Developed  for 
an  Existing  Product  Line — By  providing  a software  replica  of  a device 
which  does  not  currently  exist,  virtual  machine  systems  can  be  of  signif- 
icant value  in  performance  evaluation  studies  and  in  software  development 
work.  Since  the  VMM  can  guarantee  good  program  performance,  it  is 
possible  using  VM  techniques  to  run  highly  complex  test  or  benchmark 
programs  to  determine  the  quality  of  the  software  support.  Since  virtual 
machine  systems  usually  provide  good  tools  for  evaluating  performance 
of  software  running  on  VMs,  it  is  easy  to  perform  these  evaluations.  It 
also  becomes  easier  to  test  new  error  handling  routines  while  running  on 
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a VM  since  the  VMM  may  simulate  errors  on  the  device.  If  the  develop- 
ment of  new  peripherals  is  a frequent  activity  of  an  organization,  it  may 
be  possible  to  use  a higher  level  language  to  describe  the  device  and  com- 
pile the  VMM  virtual  I/O  device  support  directly  from  this  description. 

The  VMM  support  for  the  new  device  might  be  mapped,  partitioned,  or 
simulated  depending  upon  the  device's  degree  of  departure  from  existing 
designs. 

Scenario  2.  A New  Peripheral  is  Introduced  into  a Computer  System  and 
There  is  No  Software  Support  for  It  in  the  Commonly  Used  Operating  System 
Virtual  machines  permit  the  introduction  of  a new  peripheral  device  into 
a system  in  which  the  operating  systems  do  not  support  the  device.  Thus, 
VMMs  allow  an  installation  to  take  advantage  of  new  peripheral  technology 
without  rewriting  the  operating  system's  device  support  package.  The 
VMM  continues  to  provide  the  illusion  of  some  device  which  the  operating 
system  already  supports  while  in  reality  it  uses  the  new  device  instead. 
Depending  upon  the  similarities  between  devices,  the  support  might  be 
mapped,  partitioned,  or  simulated.  Since  today's  modern  operating 
systems  are  enormously  complex,  it  is  often  preferable  to  introduce  (the 
actual)  support  for  the  new  device  in  the  VMM  which  is  smaller,  simpler, 
and  easier  to  debug. 

The  technique  has  been  used  successfully  in  the  CP-67  system.  IBM  2314 
disk  units  were  first  introduced  into  the  system  at  the  VMM  level  while  the 
operating  system  (i.  e.,  CMS)  continued  to  manipulate  IBM  2311s  as  virtual 
I/O  devices.  The  "mini-disks"  were  supported  as  partitioned  mapped 
devices. 
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Scenario  3.  A New  Peripheral  is  Being  Introduced  into  a Computer  System 
and  There  is  No  Software  Support  for  It  in  the  Commonly  Used 
System;  However,  There  Exists  Some  Specialized  Stand-alone  Software 
Which  Does  Support  the  Device — Suppose  it  is  necessary  to  support  a new 
device  which  neither  the  predominate  operating  system  nor  the  VMM  is 
able  to  support.  Suppose  further  that  some  other  special  purpose  operating 
system  is  able  to  support  that  device.  If  standard  techniques  exist  for 
communicating  information  between  two  virtual  machines  running  under 
the  same  VMM,  it  is  possible  for  the  predominant  operating  system  to 
use  the  device  by  sending  access  requests  to  the  special  purpose  operating 
system  via  the  VMM's  communication  mechanism.  Thus  no  changes  have 
to  be  made  to  any  of  the  systems.  Furthermore,  the  process  of  debugging 
the  device  handling  routines  in  the  special  operating  system  can  only 
affect  that  system  and  cannot  cause  the  predominant  operating  system  to 
crash.  These  techniques  were  used  very  effectively  by  MIT  Lincoln 
Laboratory  in  debugging  and  introducing  support  for  the  ARPANET  in  the 
CP-67  system. 


f 
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A similar  scenario  arises  when  it  is  necessary  to  run  test  and  diagnostic 
software  for  a new  device  before  that  software  has  been  integrated  into 
the  commonly  used  operating  systems.  In  this  case  the  standard  operating 
system  runs  on  one  virtual  machine  while  the  Test  and  Diagnostic  Monitor 
runs  on  another.  There  is  no  need  to  communicate  between  virtual  machines 
in  this  example  since  the  (stand-alone)  Test  and  Diagnostic  Monitor  is 
controlled  through  the  virtual  operator's  console  of  the  VM  it  is  running 
on. 


! ; 
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SOFTWARE  EXTENSIONS:  THE  SERVICE  MACHINE 


The  VMM  under  evaluation  in  this  effort  represents  an  implemer*  r ‘.ion  of 
functionality  which  was  known  to  be  a partial  fulfillment  of  the  go  Jjg  of  a 
long-term  project.  Therefore,  there  are  several  areas  of  functional  exten- 
sion which  are  known  to  be  desirable  from  several  points  of  view.  Among 
these  are  extensions  for  sharing  peripheral  equipment,  including  front  end 
processors,  enhanced  system  console  functionality  to  permit  dynamic 
changes  in  virtual  machines  being  run  and  their  real  resource  assignments, 
an  ability  to  dynamically  swap  virtual  machines  so  that  several  can  be 
multiplexed,  improvements  to  the  treatment  of  timer  and  clock  information 
so  that  each  virtual  machine  can  have  a correct  time-of-day,  and  imple- 
mentation of  the  service  machine  concept  as  a vehicle  to  greatly  enhance 
the  VMM  functionality  for  those  services  which  can  tolerate  the  internal 
delays  associated  with  dispatching  a virtual  machine  (i.  e. , the  service 
machine). 


The  main  recommendations  of  software  issues  which  result  from  this 
analysis  are  that: 

• Performance  degradation  due  to  input/output  operations  is  sub- 
stantial as  discussed  above  and  needs  careful  analysis  in  the 
design  of  this  or  any  other  VMM. 

• The  areas  of  missing  functionality  identified  above  are  important 

in  order  for  a VMM  to  be  of  practical  value  for  most  areas  of 

application. 

i 
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In  this  section  we  present  an  implementation  approach  for  the  virtual 
machine  monitor.  The  approach  derives  from  desires  to  keep  the  perma- 
nently resident  VMM  code  as  small  as  possible  and  to  make  use  of  already 
existing  software.  In  particular,  much  of  the  functionality  required  within 
the  VMM  can  be  found  in  either  the  Multics  or  GCOS  operating  system. 

Two  important  examples  of  this  necessary  functionality  are  an  interactive 
user  interface  and  a named  information  storage  system. 

Figure  17  is  one  example  of  a VMM  implemented  using  Multics  to  provide 
support  for  the  resident  part  of  the  VMM.  This  figure  shows  a stylized 
view  of  a computer  system  memory.  There  is  a permanently  resident  area 
which  is  part  of  the  VMM.  The  Multics  operating  system  is  running  in  one 
virtual  machine,  and  two  different  GCOS  operating  systems  are  operating 
independently  in  two  additional  virtual  machines. 

Within  the  Multics  virtual  machine  there  are  many  processes  running 
which,,  in  this  example,  fall  into  two  categories:  service  machine  processes 
(SMP)  and  user  processes.  The  SMPs  comprise  the  rest  of  the  VMM  be- 
yond the  resident  part  of  the  VMM  (RVMM).  In  general  there  is  one  SMP 
per  active  virtual  machine  whether  or  not  that  VM  is  physically  resident 
in  primary  memory.  Further,  there  are  additional  SMPs  working  on 
behalf  of  the  RVMM  to  perform  overall  functions  for  the  VMM  (e.  g. , VMM 
operator  console,  I/O  spoolinjg  activities,  etc.). 

Within  the  Multics  virtual  machine  are  also  shown  user  processes.  These 
are  shown  only  to  suggest  that  the  Multics  system  could  be  used  to  support 
normal  Multics  service  for  a user  community  in  addition  to  the  SMPs. 


However,  some  system  usage  environments  might  choose  to  run  two 
Multics  VMs  because  of  the  system  isolation  which  that  approach  would 
achieve. 

The  virtual  machine  which  is  running  the  operating  system  supporting  the 
resident  part  of  the  VMM  is  called  the  service  virtual  machine  or  service 
machine  (SM)  for  short.  This  virtual  machine  has  some  constraints  placed 
upon  it  by  RVMM  due  to  the  special  nature  of  the  SM.  For  example,  RVMM 
knows  how  to  bootstrap  the  SM  into  operation  at  system  initialization  and 
will  not  allow  the  SM  to  be  halted. 

The  division  of  function  between  RVMM  and  SM  is  based  primarily  on  the 
points  raised  at  the  beginning  of  this  section.  In  particular,  functionality 
is  in  general  provided  by  an  SMP  if  at  all  possible  since  doing  so  allows 
development  of  software  which  can  make  use  of  the  rich  support  environ- 
ment of  an  operating  system.  Only  functions  which  have  great  performance 
importance  or  logical  necessity  are  included  in  the  RVMM.  For  example, 
I/O  support  would  be  included  in  RVMM  but  VM  start/ stop  support  and 
other  "console  functions"  would  be  provided  through  SMPs. 

The  interface  between  the  service  machine  and  the  RVMM  is  provided  as 
a virtual  device.  In  particular,  choosing  a communications  device  as  the 
vehicle  for  messages  between  RVMM  and  SM  has  the  benefit  of  allowing 
complete  physical  decoupling  of  the  SM  and  RVMM  so  that  the  SM  might 


RVMM  sends  messages  to  SM  by  simulating  an  interrupt  to  the  virtual 
machine  running  SM  for  the  appropriate  communications  line  and  device. 
Likewise,  SM  sends  messages  to  RVMM  using  the  normal  operating  system 
1/ O code  for  the  communications  device  and  line  which  is  appropriate.  The 
RVMM  recognizes  this  I/O  request  and  directly  consumes  the  message. 

HARDWARE  EXTENSIONS 

The  main  hardware  issues  in  the  design  of  any  VMM  relate  to  the  question 
of  how  much  of  the  system's  overall  resources  are  used  in  supporting 
virtual  machines.  Each  real  resource  of  a computer  system  (processor, 
memory,  devices,  channels,  switches)  needs  to  be  replicated  for  each 
virtual  machine.  To  do  this,  the  hardware  must  either  directly  perform 
the  mapping  between  virtual  and  real  resources,  or  it  must  have  some 
mechanism  of  intercepting  references  to  virtual  resources,  capturing  the 
complete  state  of  the  virtual  machine,  and  passing  control  to  some  other 
agent  (e.  g, , a software  or  firmware  VMM).  This  agent  then  simulates 
the  correct  behavior  of  the  virtual  resource  and  returns  control  to  the 
virtual  machine  for  farther  execution  under  direct  hardware  control.  The 
agent  consumes  real  resources  in  performing  this  simulation  and  this 
effect  can  be  substantiated. 

In  the  VMM  under  study  it  was  shown  that  the  amount  of  real  system  re- 
sources consumed  by  the  VMM  in  processing  input/ output  operations  is  a 
significant  level  for  normal  computer  system  work  loads.  The  percent  of 
the  central  processor  needed  for  this  purpose  ranged  from  10  to  30  for 
normal  GCOS  work  loads  and  somewhat  higher  for  Multics. 
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Direct  hardware  support  for  input/ output  operations  is  the  single  most 
important  potential  for  decreased  VMM  overhead.  Such  support  would  not 
be  required  for  all  I/O  devices.  A study  of  the  device  usage  shows  that 
disk  and  tape  units  have  the  most  potential  for  reducing  VMM  overhead  since 
I/O  operations  occur  most  frequently  on  these  units. 

Direct  support  of  unit  record  equipment  is  not  needed  especially  since  it 
is  desirable  for  this  equipment  to  be  shared  by  all  virtual  machines  via 
direct  VMM  control  of  the  device  and  spooling  of  the  records. 

Further  investigation  into  the  architectural  approaches  to  communications 
front  end  processor  support  is  necessary  before  the  question  of  appropriate 
hardware  support  for  virtualization  can  be  answered.  The  importance  of 
such  support  is,  of  course,  dependent  on  the  types  of  work  loads  that  might 
be  processed. 

Another  area  in  which  hardware  support  is  of  great  importance  is  that  of 
main  memory  mapping.  The  present  VMM  statically  allocates  the  full 
real  memory  required  by  a virtual  machine.  For  several  classes  of  use 
of  virtual  machines,  it  would  be  desirable  to  have  more  dynamic  control 
of  the  mapping  between  virtual  and  real  memory  resources.  In  particular, 
a block  assignment  or  paging  mechanism  could  be  investigated  for  its  effect 
on  VMM  overhead  for  different  classes  of  work  loads. 


EVOLUTION  OF  HONEYWELL  COMPUTER  PRODUCTS 


The  future  development  of  Honeywell  computer  equipment  depends  upon 
the  direction  imposed  by  industry  plus  the  technology  available  from  the 
research  and  development  groups  both  within  Honeywell  and  in  the  academic 
community.  Honeywell  has  a history  of  capitalizing  on  advanced  develop- 
ment which  is  a result  of  their  position  as  a leader  in  digital  technology. 

External  Influences 


The  major  influences  on  the  future  products  come  from  two  sources: 
industry  direction  and  architectural  influences.  The  industry  needs  for 
distributed  computing  power  at  the  point  of  need,  working  on  common 
centralized  data  bases,  are  reflected  by  the  Distributed  Systems  Environ- 
ment announced  recently  by  Honeywell.  This  environment  allows  many  of 
Honeywell's  computer  products  to  cooperate  in  new  ways  to  achieve  the 
desired  goal.  The  Level  6 minicomputer  can  be  used  as  a local  batch 
processor,  a remote  job  entry  device,  or  a message  coordinator  in  a 
larger  network.  When  combined  with  the  Level  66  GCOS  machines  and 
the  Level  68  Multics  system,  a network  of  considerable  flexibility  can  be 
constructed. 

The  architectural  influences  come  from  our  understanding  that  computer 
architecture  comprises  the  rules,  standards,  protocols,  and  guidelines 
that  govern  the  design  and  development  process.  Also  included  is  the  user 
interface  to  such  systems.  Influences  of  technology  evolution  and  appli- 
cation evolution  point  to  a comprehensive  architecture  for  the  80' s whose 
characteristics  must  be: 
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• Very  high  performance,  cost  effectiveness 

• Larger  capacity,  faster  peripheral  storage  devices 

• Utility  grade  availability 

• Support  for  large  distributed  data  bases 

• High  performance  transaction  processing 

• Easy  to  use  development  and  inquiry  systems 

• Standard  communication  links 

• Aids  for  predicting  and  evaluating  system  performance 


Increasingly  we  will  see  the  use  of  minis  as  support  to  large  systems, 
message  switches,  communication  processors,  and  terminal  controllers. 
Many  of  our  large  systems  provide  the  needed  characteristics.  What  is 
needed  for  the  80' s is  a means  to  interconnect  them  and  to  provide  cross 
compatibility  for  user  programs.  This  requires  common  and  consistent 
interfaces. 

Internal  Influences 

Honeywell  has  recognized  the  need  of  providing  a continued  flow  of  advanced 
concepts  into  the  production  divisions  and  has  stimulated  such  research  in 
several  ways. 

First,  each  computer  division  in  the  U.S.  and  Europe  maintains  an 
advanced  development  group  in  hardware  and  software.  These  groups  are 
intimately  familiar  with  current  products  and  constantly  seek  new  ways  to 
use  them  and  modifications  which  can  improve  performance  and  reliability. 


66 


Typical  of  the  work  of  these  advanced  development  groups  is  the  VMM  now 
at  RADC.  This  software  coupled  with  slightly  modified  hardware  opened 
up  a new  avenue  of  applications  for  the  Multics  and  GCOS  systems.  The 
VMM,  though  not  available  as  a standard  product,  has  proven  influential 
in  new  products  as  will  be  shown  later. 

Secondly,  the  Systems  and  Research  Center  of  the  Aerospace  and  Defense 
Group  has  carried  on  an  active  program  in  distributed  computing  research. 
Their  efforts  have  been  aimed  directly  at  the  government  market  place 
for  applications  such  as  flight  and  weapon  control  systems,  command  and 
control  systems,  and  certifiably  secure  computer  programs.  In  fact,  it 
is  SRC  that  continued  the  VMM  work  represented  in  this  document. 

Third,  it  is  recognized  that  computer  architectures  of  the  1985  to  1990 
time  frame  will  require  significant  research  and  development  prior  to 
production.  As  a general  response  to  that  need.  Honeywell  formed  the 
Corporate  Computer  Sciences  Center  in  Minneapolis  and  directed  that  a 
program  to  address  the  architecture  needs  of  the  late  80's  be  started.  This 
architecture  work  is  responsible  for  interfacing  with  the  research  communi- 
ty, identifying  promising  ideas,  and  feeding  these  ideas  into  the  planning 
and  advanced  development  groups.  The  VMM  is  being  considered  in  the 
course  of  that  study. 

The  Role  of  the  VMM 

The  virtual  machine  monitor  was  an  early  attempt  at  extending  the  power  of 
the  hardware /software  base.  Its  minimal  functionality  is  limiting  in  signifi- 
cant ways,  yet  the  concept  points  to  an  important  fact:  if  VMM  performance 
is  improved,  the  power  gained  should  be  exploited. 
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The  advanced  development  processors  currently  on  the  drawing  boards 
include  a concept  referred  to  as  the  hypervisor.  The  hypervisor  is  a VMM 
in  hardware /firmware  with  some  modifications.  Honeywell  has  recognized 
that  the  two  personalities  of  Level  66  and  Level  68  could  be  put  to  produc- 
tive use  if  a dual  personality  machine  could  be  designed.  Some  general 
benefits  of  such  a multicomputer  or  hypervisor  approach  are: 

• Ease  of  moving  users  between  systems 

• Coexistence  of  different  systems 

• Expanded  development  and  testing  capabilities 

• Increased  capabilities  with  multiple  copies  of  a limited  system 

• Increased  availability  with  multiple  copies 

• Common  interfaces 

• Better  test  and  development  facilities 

These  benefits  are  directly  in  line  with  what  has  been  learned  about  the 
existing  VMM.  It  is  conceivable  that  the  hypervisor  approach  may  be 
reflected  in  future  Honeywell  large  scale  processors,  though  changing 
market  demands  could  cause  this  strategy  to  change. 
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SECTION  V 

RECOMMENDATIONS 


FUTURE  VMM  RESEARCH 

Distributed  Systems  of  Virtual  Machines 

The  feasibility  of  the  virtual  machine  concept  has  been  proven.  Further- 
more, the  usefulness  of  supporting  communication  among  clusters  of 
virtual  machines  has  been  described  by  S.E.  Madnick.1  However,  tech- 
niques and  interfacing  standards  still  need  to  be  determined  for  inter- 
virtual  machine  communication  and  synchronization  especially  when  the 
VMs  are  supported  by  physically  distributed  hardware  systems. 

Specialized  Virtual  Machines 

In  a system  of  logically  distributed  processing,  it  might  be  expected  that 
certain  systems  in  the  network  will  be  performing  specialized  functions 
on  behalf  of  the  other  member  systems.  An  example  of  this  is  the  system 
which  controls  a data  base  to  be  shared  among  all  member  systems.  The 
issue  to  be  investigated  is  the  nature  of  the  interface  between  such  a 


S.E.  Madnick  and  C.  Lam,  "Composite  Information  Systems--A  New 
Concept  in  Information  Systems, " Report  No.  35,  Center  for  Information 
Systems  Research,  Massachusetts  Institute  of  Technology,  1978. 
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specialized  virtual  machine  and  a VMM.  For  example,  the  VMM  might 
support  high  level  data  base  access  primitives  which  completely  isolate 
the  specialized  virtual  machine  from  physical  device  characteristics. 


Incremental  System  Extension  and  Integration 


The  virtual  machine  interface  to  other  virtual  machines  in  a distributed 
system  of  virtual  machines  is  well  defined  and  enforced  by  the  VMM.  This 
well  defined  interface  might  be  used  as  the  point  for  formal  definition  of 
a particular  virtual  machine  function.  Changes  in  binding  between  a 
virtual  machine  and  the  physical  hardware  used  to  support  it  could  be 
made  without  requiring  revision  of  the  software  within  a virtual  machine. 
Likewise,  a cluster  of  virtual  machines  supported  by  one  fast  processor 
could  be  moved  to  a hardware  support  base  of  several  (slower)  processors 
without  software  revision. 


An  additional  topic  for  investigation  under  this  subject  is  the  feasibility  of 
replacing  the  formal  interface  between  two  particular  virtual  machines 
with  a new  formal  interface.  As  a large  system  evolved,  some  virtual 
machines  might  be  replaced  with  newer  ones  designed  to  a different  formal 
interface.  It  might  be  useful  in  such  situations  to  provide  formal  interface 
translator  virtual  machines  which  could  translate  between  different  versions 
of  a formal  interface. 


CONCLUSIONS 


The  work  performed  during  this  effort  has  been  valuable  in  that  it  demon- 
strated feasibility  of  a VMM  in  the  RADC  environment  and  identified  the 
performance  tradeoffs  which  must  be  made.  The  future  support  of  this 
VMM  by  Honeywell  is  uncertain;  however,  the  concept  and  many  of  the 
design  features  have  been  integrated  into  existing  and  planned  products. 

RADC  has  played  a useful  role  in  supporting  the  benefits  of  the  virtual 
machine  approach  for  certain  types  of  problems  and  this  work  might  well 
continue.  The  most  fruitful  area  for  further  research  appears  to  be  in 
the  design  of  a VMM  which  is  more  tolerant  of  I/O  activity.  It  is  doubtful 
that  the  existing  prototype  VMM  could  be  improved  sufficiently  to  allow  it 
to  function  effectively  in  a production  environment.  The  prototype  has 
served  its  purpose  of  demonstrating  functionality  and  feasibility. 
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. LRIOT  ■ IDLE  TIME  ■ C8UF(28> 

00000110 

B 

241 

a 

. CR > OT  CURRENT  SC  CLOCK  VALUE  FROM  VMM- 

CBUF (26 >000001 20 

B 

247 

■ 

. 5RTWT  n • OF  TIMES  IDLE  > CBUF (34) 

00000130 

B 

no 

■ 

igBTM  ■■  T9TAU  RUrAJCMES  ■ CBUF<?8) 

00000140 

6MUMB  • 2 777T,  ACTIVITY  • ■ 01 , REPORT  CODE  ■ 06,  RECORD  COUNT  ■ 00402 


II 

1040 

1046 

IMP 


. CRUSE ♦ 1 "VMM  PROC.  TIME  PROM  VMM 
.CRUSE*2-VMM  IDLE  TIME  FROM  VMM 


• C8UF (416) 
C8UF (417) 


■ .CRU6EQ.MULTIC6  VP  TIME  FROM  VMM  ■ C»UF<416) 


00000160 
00000170 
00000160 
666661  AD 
00000200 
00000210 
00000220 


CHARACTER  DATE 


| MiBB  looSs/daos/  , c6uf  < 666  > , 66u»1t/6686> , it aT 

DATA  PULHR6/ 260400000. /, PUL8EC/64000 . / 

REAL  MICHRS/3600000000.'/,MIC8EC/1000000.  / 


66666230 

00000240 

00000290 

>0000260 


REAL  ITIME, IOVH, I IDT, IRTHRS, I0C9P, IVN1P, IVMMIOL, IMTXP^ 
REAL  LTIME, LOVH, LI DT, LRTHRS, LOCSP, LVMMP, LVMMI DL. LMTXP 
REAL  CTIME, COVH, Cl DT,CRTHRS, C0C9P, CVMMP, CVMMI DL, CMTXP 
IHTE6ER  CT1R.CTCN.CTWT.CT0S  


INTEGER  CRT I NT 

INTEQER  PI ,P2,P3,P4,P6,P6,P7,P6 
INTE6ER  TCN.T1R.TD6.TWT 


30270 
00000260 
00000290 
00000300 


00000300 

00000310 

00000320 

-ggftMWQ 


00000340 

00000380 

00000360 

00000370 


CALL  6LEEPX ( 30 ) 
INITIALIZE  ALL  C0UNTER8 


CALL  'TtMEX(6ATt/fg5*X-1 

CALL  HCHOV(AOOR6,C6UF, COUNT. 8TAT) 
IFI8TAT. NE. 0)60 TO  200 

l?ISUtii?OTiTWf*l/f<u** 

ITCN-C6UF12) 

I OVHaFLOAT ( COUP ( 6 ) I/PUL6EC 

IF( IRTINT. NE. 0160T0  10 
CALL  ROCLCKd1.lt> 

IRTI 


00000360 

00000390 

00000400 

30000410 


OOOOt 

6oooc 


36426 
00000430 
00000440 
00000480 
00600460 
00000470 
00000400 
00000490 


.6 


66666boo 

00000810 

00000820 

00000830 

66666B46 

00000880 

00000860 

00000870 

'66BB6S66 


CALL  AOJSCCI IRTINT) 

I RTHRS-FLOAT 1 1 RT I NT > /Ml 

msasaiK 


Ju  j l1] atil  ^ 


SHIS  PAGES  IS  BEST  QUALITY  FlUCtICABM 
VMH’OOPY  rUKHLSHHD  SOLDO • 


IGCSF  .OAT ( CBUF (418) ) /M I CSEC 
I VMMP*FL0AT(CBUF(416> )/MICSEC 
I VMM I DL=FLOAT ( CBUF( 4 1 7) ) /Ml CSEC 
I MTXP*FLOAT l CBUF ( 4 1 8 ) ) /M I CSEC 

WRI TE ( 06, 1 000) DATE 


WR 1 TE  ( 07, 1 1 00 ) DATE 

WRI TE( 07,  1 110) I TIME,  IRTHRS, I VMM I DL , IVMMP,  IOCSP, IMTXP 


RESET  COUNTERS  FOR  LOOP  CALCULATIONS 


LTI ME* I T I ME 

I TIRilTI 


LTCN* I TCN 
LOVH* I OVH 
L I OT * I IDT 


WRITE (39, 1300) DATE 


CONTINUOUS  LOOP  TO  SAMPLE  HCM  CELLS 


CONT I NUE 
CALL  SLEEPXMB) 

CALL  TIMEX (DATE, TODPUL) 


I F(STAT . NE . 0 )90T0  210 

CT I ME* FLOAT ( TODPUL ) /PULHRS 
( 1 ) 


CTCN*CBUF( 2) 

COVH*FLOAT ( CBUF ( 9) ) /PULSEC 
Cl OT  *FLOAT ( CBUF ( 28) ) /PULSEC 


00000S90 

00000600 

00000610 

00000620 

00000630 

00000640 


■•T»T*T*T»T-}-r« 


00000660 

00000670 

00000660 


00000700 

00000710 

00000720 


00000740 

000007S0 

00000760 


■ •T*T*T*T •>#/• 


LTWT ■ 1 TWT 

00000760 

LTDS* 1 TDS 

00000790 

LOCSP* IOCSP 

00000600 

i*I*T*I*I*I  • H*I 


LVMMI DL* 1 VMMIDL 

00000620 

LMTXP* IMTXP 

00000630 

C 

00000640 

i*x*i*i*i*i-i-r*i 


00000660 

00000670 

00000860 


00000900 

00000910 

00000920 


00000940 

000009B0 

00000960 


I'/i'm-j 


00000960 

00000990 

00001000 


*i*l*l*Jl*JI 


00001020 

00001030 

00001040 


Tillld  ~\w.JLw  4 w 


■1*1*1*J  1*1*1 


I F ( CRT I NT . NE . 0 ) OOTO  110 
CALL  RDCLCK (II, 12) 

CRT  I NT* I 2 


00001060 

00001070 

00001060 


k ■■  ■ » l*J 


»l#l*I*Jl*i’I* 


CALL  ADJSCC( CRT I NT) 

CRTHRS* FLOAT ( CRT  I NT ) /Ml CHRS 
CTWT*CBUF ( 34 ) 


00001100 
00001  110 
00001120 


COCSP*FLOAT ( CBUF (4 1 B) ) /Ml CSEC 
CVMMP*FLOAT ( CBUF (416) ) /MI CSEC 
CVMMI DL*FLOAT ( CBUF ( 4 1 7 ) ) /Ml CSEC 


00001 140 
00001  1 BO 
00001160 


00001160 

00001190 

00001200 


CALCULATE  VALUES  SINCE  INITIALIZATION 


•TvMTITT  J • 


ZHIS  PACES  XSJ 

raon  bopyyus 


tSCTD  K>DDC 


T2-(F  »T ( CRT INT-1RTINT) /M I CSEC ) ♦ .049 

1 2- T2* 10. 

T3-CI DT- I 1 DT  ♦ .049 

1 3- T3* 1 0. 

PI « ( T3/T1  ♦ . 0049X100. 

P2- < T3/T2  ♦ . 0049) * 1 00 . 


if  ■urt-i 


I 4-T4* 1 0. 

P3-  < T4/T1  + .0049X100. 
P4-  ( T4/T2  ♦ . 0049) * 100. 


CALCULATE  VALUES  SINCE  LAST  SAMPLE 

T5- ( ( CTI ME-LTIME) »3600 . ) * .049 
1 


T6- (FLOAT ( CRTl NT-LRTI NT) /Ml CSEC)  ♦ .049 
I6-T6-10. 

T7-CIDT-LIDT  ♦ .049 

I fmT-7*  1 n 


P8-  ( T7/T5  ♦ .0049X100. 
P6- ( T7/T6  ♦ .0049X100. 
T8-C0VH-L0VH  ♦ .049 

I AaTA* 1 n 


P7-  ( T6/T5  ♦ .0049X100. 
P8-(T8/T6  ♦ .0049X100. 


TCN-CTCN-LTCN 

TIR-CTIR-LTIR 

TOS-CTDS-LTDS 


WR I TE ( 06 , 1 0 1 0 > CT I ME , C I DT , COVH , CTCN , CT I R , CTOS , CTWT 
WRI TEC 42, 1 21 0) CTI ME, 11, 12, 13, PI ,P2, 14.P3.P4, 


B1-T2 

1X6X10. 

62-CVMMIDL-IVMMIDL  ♦ .049 
12-62*10.  


PX  (62/61  ♦ .0049)  *100. 
B3-CVMMP- I VMMP  +0.049 
13-63*10. 

♦ , 0049) » 1 


64 ■ CQCSP - I OCSP  ♦ .049 
14-64*10. 

P3-  <64/61  ♦ .0049X100. 

♦ .049 


16-66*10. 

P4-C68/B1  ♦ .0049X100. 


i*fj  *.i  i mri  m .■  •7.1  f.wj  i : w m w.v  % m-  j. j » i j m * 


B8-T6 

16-66*10. 


»II»I 


17-67*10. 

P6- (67/66  ♦ .0049) *100. 
68 - C VMMP -L VMMP  ♦ .049 
I listlio 


P6-( 68/66  ♦ .0049) *100. 


00001230 

00001240 

00001260 

00001260 

00001270 

00001260 


00001300 

00001310 

00001320 


l*IaI*I«l*<f<Ial 


00001340 

00001360 

00001360 


00001360 
00001390 
00001400 
1 


00001420 

00001430 

00001440 


00001460 

00001470 

00001460 


i*i 


00001500 

00001510 

00001520 


00001540 

00001550 

00001560 


00001620 

00001630 

00001640 

00001650 


00001660 
00001670 
00001660 
000169 


00001700 
00001710 
00001720 
00001 


00001 740 
00001750 
00001760 


00001760 

00001790 

00001600 


IaI*I’IclK-lKa] 


00001620 

00001630 

00001640 


1*I*I*1*11 1-1*1 


00001860 
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WR I TE 1 07 , 1 1 1 0 > RT I ME , CRTHRS , CVMM I DL , CVMMP . CGCSP , CMTXP 
WR1T£<39, 1310)RTIME, I 1, 1 2 , P 1 , I 3 , P2 , 1 4 , P3 , 1 3 , P4 , 

« 16,  I 7, P5,  I 8, P6,  1 9,  P7,  I 1 0,  P8 


RESET  COUNTERS  FOR  NEXT  LOOP 
LTIME»CTIME 


00001940 

00001950 

00001960 


00001980 

00001990 

00002000 


LTCN-CTCN 

00002020 

LOVH-COVH 

00002030 

LRU  NT  ■ CRT  1 NT 

00002040 

LTWT-CTVT 

LTDS-CTOS 

LVMMIOL-CVMMIDL 

LVMMP-CVMMP 

LOCSP-COCSP 

LMTXP-CMTXP 


cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc 


ii-j  m . 


200  WRITE (06, 1S001STAT 

STOP  200 


STOP  210 


I . I M.H  IV.W/.1  w.w  Ic.n  M'  1*1' ■ : M M ■ i-J  immWA-J 


• “ TIME  I OLE -SEC  0VERHD-SEC  CONNECTS 
» "INTERRUPTS  DISPATCHES  9T1MES- I OLE" , ///) 

1010  FORMAT (1X,F6.3,F9. 1 , 3X, F9. 1 , 1 X, 1 9, 3X. 1 9, 3X, 1 9, 4X, I 9) 


1100  FORMAT ( 1H1 , " ACTUAL  DATA  FROM  VMM  FOR  ",A6,//, 

■ " RTIME  SC-HOURS  VMM- IDLE-SEC  VMM-PROC-SEC" , 
« " 9CS-PR0C-SEC  MTX-PROC-SEC" , ///> 


1200  FORMAT <1  HI,"  OCOS  DATA  VALUES  FOR  DELTA  TIME  INTERVAL  FOR 
» AS./.SX, "(LAPS, IDLE.OVH  ARE  IN  1/10  SECOND  UNITS)",//, 


• 20("-"), "DATA  SINCE  LAST  SAMPLE" , 20 ( /. 

■ IX, 6X,"  ©LAPS  RLAPS  IDLE  10  SR  OVHD  XG  XR 

■ "OLAP  RLAP  IDLE  XO  XR  OVHD  XO  XR", 


Lit 


1210  FORMAT ( IX,  FS.  3,  21 6,  2(  1 6,  21  3) , 3X,  21 S,  2(  I S,  21 3) , 4 1 

% 

1300  F0RMAT< 1M1 , M VMM  DATA  VALUES  FOR  DELTA  TIME  INTERVALS  FOR 

i / tA 


00002060 

00002070 

000020SO 

QQQQZQ9Q 

00002100 

00002110 

00002120 

00002130 

00002140 

00002150 

00002160 


00002190 

00002190 

00002200 

' ~;1f 


00002220 

00002230 

00002240 


00002260 

00002270 

00002290 


00002300 

00002310 

00002320 


L'i'm 


00002340 

000023S0 

00002360 


[•wmw] 


000023S0 

00002390 

00002400 


00002420 

00002430 

00002440 


■ 


iv/i ; i x w- .161 
rTrai 


, RTIME  * , 13( *«* ) , "OATA  SINCE  START* , 14( "•"), 3X 
* 7( " *" ) , "DATA  SINCE  LAST  SAMPLE" , 7( "-"), /, 

« IX, SX,*  RLAPS  VI OLE  X VOVHD  X OCSVP  X MTXVP  X 


rAvm  *s «r/i« 


•Tir.i.Trriii 


00002470 

00002460 


,c  00002S1 0 


1SOO 

FORMAT ( 

" CORE  MOVE  ERROR  - STATUS  ■ ",  14) 

1 V 

00002820 

ENO 

00002830 

1 

GMAP 

NDECK 

00002840 

TTL 

HCMOV  - HARD 

CARD  MOVE  SUBROUT 1 NES77021 0 

00002880 

TTLOAT 

00002860 

epitp 

ON 

00002870 

SYMOEF 

HCMOV 

00002860 

SYMOEF 

SLEEPX, TIMEX 

00002890 

SYMOEF 

RDCLCK , AOJSCC 

00002600 

* 

00002610 

« 

CALL  HCMOV ( FROM , TO , COUNT , STAT ) 

00002620 

B 

00002630 

* 

CALLING 

ARGUMENTS 

00002640 

* 

FROM 

■ ADDRESS  OF  WORD  WHICH  CONTAINS 

00002680 

* 

ABSOLUTE  STARTING  ADDRESS  IN  16-38 

00002660 

* 

TO  * 

AOORESS  OF  RECEIVING  BUFFER 

00002670 

a 

COUNT 

> ADDRESS  OF 

WORD  WHICH  CONTAINS 

00002660 

a 

NUMBER  OF  WORDS  TO  MOVE  IN  16-38 

00002690 

a 

00002700 

a 

STATUS 

RETURNS 

00002710 

• 

STAT 

■ 0 - ALL  OK 

00002720 

a 

3TAT 

« 1 - COUNT  GREATER  THAN  812 

00002730 

a 

OR  ZERO 

OR  NEGATIVE 

00002740 

a 

STAT 

* 2 - TO+COUNT 

PAST  USER'S  CORE 

00002780 

a. 

STAT 

* 3 - TO  IS  BELOW  USER'S  CORE 

00002760 

a __ 

STAT 

» 4 - FROM  IS 

NOT  IN  HCM  <<64K) 

00002770 

a 

00002760 

HCMOV 

SAVE 

0.2,3, 4. 8,6,7 

ENTRY-SAVE  XO, X2, X3, X4 , X8. X6, X7 

00002790 

STA 

AR 

SAVE  A REGISTER 

00002600 

EAXO 

8.1* 

GET  ADDRESS  OF  "STAT" 

00002610 

STXO 

STAT 

SAVE  IT 

00002620 

STZ 

STAT, 1 

SET  "STAT"  TO  ZERO 

00002630 

EAXO 

2,  1* 

GET  ADDRESS  OF  "FROM" 

00002640 

— 

_U4LQ 

0.0 

GET  "FROM"  - MUST  BE  ABSOLUTE  ALREADY 

00002680 

. 

CMPXO 

*0200000, DU 

IS  FROM  < 64K  (IN  HCM) 

00002660 

TRC 

ER4 

NO  - ERROR  4 

00002670 

STXO 

FROM 

SAVE  "FROM" 

00002660 

EAXQ 

3.1* 

GET  ADDRESS  OF  "TO" 

00002690 

TMI 

ER3 

MINUS  - ERROR  3 

00002900 

STXO 

TO 

SAVE  1 T 

00002910 

_ 

EAXO 

4,1* 

GET  AOORESS  OF  "COUNT" 

00002920 

LXLO  

0.0 

GET  "COUNT" 

00002930 

1 

TMOZ 

ER1 

MINUS  OR  ZERO  - ERROR  1 

00002940 

M 

CMPXO 

*01001 , DU 

CHECK  "COUNT" 

00002980 

H 

TRC 

ER1 

TOO  LARGE  - ERROR  1 

00002960 

n 

STXO 

COUNT 

SAVE  "COUNT" 

00002970 

•j 

LDX2 

FROM 

GET  "FROM"  AGAIN 

00002900 

1 

SOAR 

BAR 

SAVE  OUR  BASE  ADDRESS  REGISTER 

00002990 

► 

a 

INHIB 

ON 

DON'T  GET  INTERUPTED  DURING  MOVE 

00003000 

> 

• 

MME 

.EMM 

•••ENTER  MASTER  MODE  •«• 

00003010 

fl 

ASXB 

TO,  6 

FROM  ABSOLUTE  "TO"  BY  ADDING  LAL 

00003020 

1 MR 

LOA 

** , DU 

GET  BAR 

00003030 

H 

I 

ANA 

■0777, DU 

GET  • OF  812  BLOCKS 

00003040 

ALS 

9 

MOVE  IT  OVER 

00003080 

STXB 

1.  1C 

SAVfe  LAL ’ ; 

66006660 

EAX3 

**,  AU 

FORM  UPPER  LIMIT 

00003070 

SBX3 

1 , DU 

SUBTRACT  ONE 

00003060 

L0X4 

TO.  6 

GET  "TO" 

00003090 

ADX4 

COUNT, 6- 

ADO  1 N" COUNT* 

60003100 

STX3 

1,  1C 

SAVE  UPPER  LIMIT 

00003110 

> 

a 

CMPX4 

**,  DU 

PAST  OUR  CORE  LIMITS 

00003120 

* 

TRC 

ER2.S 

YES  - ERROR  2 

00003130 

c 

2 

LOX3 

TO,  6 

NO  - GET  ABSOLUTE  "TO" 

00003140 
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IBIS  PASS  IS  BIST  QUALITY  PRACT1CABU 
' JRQM  OQPY  fUraUSHSD  TO  DDC 

LOX4 

COUNT,* 

OET  "COUNT" 

000031  BO 

LOA 

0.2 

OET  DATA 

000031  SO 

STA 

0,3 

SAVE  IT 

00003170 

A0LX2 

1 ,0U 

BUMP  "FROM" 

000031*0 

A0LX3 

1 ,0U 

BUMP  -TO" 

00003 ISO 

SBX4 

1 , DU 

DECREMENT  "COUNT" 

00003200 

i I I 


HIT  TSS  «*1 

INHIB  OFF 
LOA  AR 


* 


STAT, I 
STAT, I 


RETURN  TO  SLAVE  MODE 
RESTORE  A REOISTER 


BUMP  "STAT" 
BUMP  -STAT" 


■\>U  JU 


0000*2*0 

00003230 

00003240 


•I*I*I*r  J 4-.* 


000032*0 

00003*70 

0000SBS0 


•T*T«T>7-T  TT-l 


BUMP  “STAT” 

00003300 

00003310 

000033*0 

000033*0 

00003400 


.00 

2,1* 

OET  NUMBER  OF  SECONDS  TO  SLEEP 

III  III 

IPY 

84000, DL 

CONVERT  TO  PULSES 

00003430 

IME 

9EWAKE 

00  TO  SLEEP 

00003440 

l.i-Ti'JJT 


OETIME 

2,1* 

3.1* 


STORE  OATE 

STORE  TIME  IN  PULSES 


•M-H-IL! 


00003470 
000034 SO 


ffliri 


00003810 

00003620 


[•X-I-H-HJ 


00003880 


ADJUST  SYSTEM  CONTROLLER  CLOCK  VALUE  RETURNEO 


I J-I. »V.1S'] 


FOR  ROLLOVER  AFFECT 
USE  ONLY  RIOHT  MOST  34  Bl TS-8600  SECONDS 
HSOSO  HAS  MICRO  SECOND  CLOCK 


DJSCC  NULL 

LOA  2,1* 

ANA  -0177777777777  USE  ONLY  RIOHT  MOST  34  BITS 


r m 


00003*701 

00003**0 


'•T-T-T' >/.!•! 


00000710 

00000720 


l+4+< 


L i.U 


LMioiM:iiJi>M:t*  lid 


5 


POS3CC 

NO.  BRANCH 

00003790 

1.0L 

YES.  SET  FLAO 

00003S00 

FLA  3 

00003610 

SCCCXT 

OONE 

00003620 

FLAO 

IF  BIT  ■ 0,  CHECK  IF  PREVIOUSLY  WAS  1 

1 00003630 

SCCCXT 

NO, BRANCH 

00003640 

Ml  ■ II  'IMIMMI— 

— I 1 II  II  1 1 

A DOER 

ADJUST  CLOCK  VALUE 

00003660 

FLAO 

RESET  FLAO 

00003070 

TEMP 

USE  ADJUSTED  VALUE 

00003660 

f ii’ it7*H  .. 

Q00Q3B8Q 

00003900 

00003910 

00003915 


R 

OCT  0 

ENO 

00003930 

00003940 

00003950 

EXECUTE 00003960 

PRIVITY 

00003970 

LIMITS  , 1 0K 

00003960 

SYSDUT  07, ORO 

00003990 

M ' I H 1 


termlnation_overseer.pl  I 08/28/78  0930.8  edt  Mon 


terminatlon_overseer • proctascli_n_loads,  ascli_n_iterations) i 

del  ascll_n_loads  chart*)! 
del  ascll_n_lteratlons  chart*)! 
del  character. value  chart7)i 
del  coda  fixed  blnt35)l 

del  cv.dac.check.  antry  tchart*),  fixed  hlnt35))  returns  tflxed  blnt35))i 
del  cv.float.  entry  tchart*),  fixed  bint 35),  float  bint27))i 
del  fpsum  Moat  blnt27)l 
del  fpval  float  blnt27)l 

del  gat.wdlr.  entry  returns! char t I A8)  aligned)! 

del  hcs.$lnl tlate  entry  tchart*),  chart*),  chart*),  fixed  bintl),  fixed  bin 
\c  ptr,  fixed  bint 35))! 

del  hcs.Smake.seg  entry  tchart*),  chart*),  chart*),  fixed  blnt5),  ptr,  fixe 

\cnt35))l 

del  1 fixed  bln! 

del  loa_  entry  optlonstvarlable) I 

del  lpm_status_segt 32)  charts)  based  ( status_ptr ) ! 

del  n_loads  fixed  blnt35)l 

del  n_lteratlons  fixed  blnt35) t 

del  null  bulltln! 

del  status_ptr  ptri 

del  substr  bulltln! 

del  termlnation.f lag_ptr  ptrl 

del  tlmer.manager.Ssleep  entry  fflxed  bint7l),  bl t < 2 ) ) « 

n_loads  * cv_dec_check_  tascii_n_loads,  code)! 
if  code  0 then  doi 

call  ioa_  t"Bad  argument  ) Input  to  termination.overseer.  Abort, 
call  hcs.Smake.seg  1 1 get.wdl r_t  ) ) , "termlnatlon_f lag",  "M,  10,  te 
\catlon_flag_ptr , code)! 
return! 
end! 

n_lterations  ■ cv_dec_check_  tascl i_n_lterations,  code)! 

If  code  0 then  doi 

call  loa_  t"Bad  argument  2 Input  to  termlnation_overseer.  Abort, 
call  hcs.Smake.seg  f tget.wdjr.t ) ) , *termlnatlon_f lag",  10,  te 

\catlon_f lag_ptr,  code)! 
return! 
end! 

status_ptr  * null! 

call  hcs.Sinltlate  l tget_wdlr_t > ) , *1 pm_status_seg",  "",  0,  0,  status. 
\c  code)! 

If  status_ptr  ■ null  then  doi 

call  ioa_  t"Unable  to  initiate  ipm.status.seg,  Abort.") I 
call  hcs_$make_seg  ( f get_wdlr_t ) ) , "termlnatlon_f lag",  "",  10,  t« 
\catlon_f lag_ptr , code)! 
return! 
end! 
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THIS  PA®  ZS  BfiST  QUALITY  PIU.CTICABU 
FROM  OOPY  7WHLSOT  TOSDQ  


do  1 ■ I to  n.loadst 

ipm_status_seg( i)  ■ “ZZZZZZZZ"! 
and  i 

do  1 ■ I to  n_ loads  I 

do  while  (ipm_status..seg<  1)  » "ZZZZZZZZ")! 
call  timer_manager..$sleep(30f  "ll^b)! 
and! 

and  i 

termlnation_f lag_ptr  ■ null!  _ „„  tn 

call  hcs_$make_s*g  ( <get_wdir_< ) ) , *t#rmination_f lag",  10, 
Sc_f lag_ptr , coda ) t 

call  timar_managar_Sslaep<30,  "U-"b)i 

do  1 ■ I to  n_loads« 

do  while  ( ipm_status_seg( 1)  » "finished") » 
call  timer_managar_$slaep(30,"l  l"b)  l 
and! 

andt 

1 psum  ■ 0. • 

do  1 ■ I to  n_loadsi 

character_valua  * substr ( lpm_status_sag( 1 ) , >«  7)1 
call  cv_float_  <charact*r_valua,  coda,  fpvalM 
fpsum  ■ fpsum  ♦ fpvall 
and  i 

call  ioa_  ( "~3/*++**  Thruput  for  ~d  loads  and  ~d  iterations  is 
\c ions  par  minute  aaa*#*3/",  n_loads,  n_itarations,  fpsum)! 

and  term in at ion_o vers ear  t 
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SECTION  1 


INTRODUCTION 

ORGANIZATION  OF  THIS  REPORT 


Since  this  document  was  produced  at  approximately  the  midpoint  of  the 
contract  term,  final  conclusions  concerning  the  Virtual  Machine  Monitor 
(VMM)  performance  cannot  be  drawn.  Experiments  are  still  underway  at 
the  time  of  this  writing.  This  report  is  an  attempt,  therefore,  to  document 
the  functional  characteristics  of  the  VMM  in  a way  that  closely  parallels  its 
observed  performance. 

Section  2 describes  the  VMM  down  to  the  module  level  of  detail.  Much  of 
this  information  has  been  taken  from  the  documentation  produced  with  the 
VMM  implementation.  The  reader  will  notice  therefore  that  it  closely 
follows  the  WELLMADE  methodology  documentation  structure  in  use  at  the 
time  the  design  was  done. 


Section  3 describes  the  design  approach  for  evolution  of  the  VMM.  In 
particular  virtual  device  support  is  treated  in  detail  for  shared  front-end 
network  processors  and  shared  operator  consoles. 

Additional  material  like  that  in  Section  3 will  be  supplied  at  the  termination 
of  the  contract,  following  more  extensive  analysis  of  the  VMM  performance 
in  live  tests. 

I 

I 
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SECTION  2 


VMM  DESCRIPTION 


FUNCTIONAL  DESCRIPTION  OF  VMM 

The  VMM  provides  two  environments  within  a single  6180  system.  These 
are  indistinguishable  from  the  normal  6000  program  environment  and  the 
6180  program  environment.  They  are  referred  to  as  virtual  machines. 
These  evnironments  are  established  by  interfaces  fabricated  of  software 
and  hardware.  When  these  are  viewed  from  an  operating  system,  they  are 
indistinguishable  from  the  real  hardware  interfaces  assumed  by  the 
operating  system  when  executing  on  a real  machine. 

The  VMM  under  consideration  supports  multiple  virtual  6000  machines  and 
one  virtual  6180  machine.  Therefore  it  is  possible  for  a single  6180  system 
to  concurrently  support  multiple  GCOS  systems  and  one  MULTICS.  Hence 
we  have  a situation  in  which  a VMM  supports  multiple  operating  systems  in 
much  the  same  way  that  a multi -programmed  operating  system  would  support 
two  (or  more)  user  jobs. 


The  hardware  interface  part  of  the  virtual  environment  consists  of  several 
hardware  changes  to  the  basic  6180  CPU,  a change  to  the  IOM  Direct  Channel 
and  a change  to  the  355  DIA  Control  Board.  The  software  interfaces  are 
supplied  by  the  VMM  and  are  the  subject  of  this  and  related  documentation. 
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The  responsibilities  of  the  VMM  are  as  follows: 

• P.artition  the  available  real  machine  resources  in  n pieces. 

One  set  is  reserved  for  the  use  of  the  VMM.  The  remaining 
resources  are  divided  to  provide  one  set  each  for  n-2  virtual 
6000  and  one  for  the  virtual  6180. 

• Protect  each  operating  system,  its  environment  and  the  VMM 
from  reference  by  the  other  operating  system  or  any  of  its 
users. 

• Provide  for  the  allocation  of  the  real  machine  processing  and 
peripheral  resources  to  satisfy  the  resource  needs  of  the 
virtual  machine  (VM). 

• Provide  for  the  orderly  start-up  of  the  VMM,  the  GCOS  and 
MULTICS. 

It  should  be  noted  that  the  partitioning  of  resources  must  apply  not  only  to 
main  memory  but  to  secondary  storage  and  peripheral  devices  as  well.  It 
must  not  be  possible  for  any  user  of  any  operating  system  to  reference  a 
virtual  machine  other  than  the  one  it  is  allocated  to  serve.  (An  anomaly 
exists  in  this  respect  with  regard  to  355s  in  version  1. 1.  See  Known 
Limitations  below. ) 

Usage  Information  for  VMM 

To  initiate  VMM  operation,  a bootload  procedure  will  be  started  by  depress- 
ing the  "boot"  button  on  the  master  console.  This  will  cause  the  VMM  to 
be  loaded  from  cards.  The  system  configuration  will  be  specified  by  tables 
internal  to  the  start-up  deck.  Once  the  VMM  has  been  started  up,  either 
or  both  of  the  virtual  machines  may  be  started  from  a console. 
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In  this  version  of  VMM  (only)  it  is  the  user's  responsibility  to  establish  the 
configuration  decks  of  the  GCOS  and  MULTICS  to  be  executed  in  such  a way 
that  they  use  a complementary  set  of  the  real  peripheral  resources  of  the 
system.  Each  operating  system  must  have  its  own  console  and  unit  record 
peripherals.  It  must  also  have  its  own  FNP  if  one  is  used  and  its  own 
magnetic  tape  handlers  and  disk  spindles.  However,  disk  and  tape  control- 
lers and  IOM  may  be  shared.  Furthermore,  the  addresses  of  the  peripheral 
devices  used  must  be  their  real  device  addresses. 

After  an  operating  system  has  been  started  from  a console,  it  will  be  used 
exclusively  by  that  operating  system.  Communication  with  the  VMM  will 
not  be  possible  again  unless  or  until  the  VMM  detects  that  the  operating 
system  has  ceased  operation.  At  this  time,  the  operating  system  may  again 
be  started  according  to  the  initial  start-up  procedure. 

Once  a virtual  machine  has  been  started,  its  interfaces  to  its  operator  and 
its  users  will  be  identical  to  those  provided  when  it  is  in  execution  on  a real 
machine.  These  are  described  elsewhere  in  appropriate  manuals. 

Resource  Objectives  of  VMM 

Version  1.1  of  VMM  shall  operate  on  6180  systems  with  memories  greater 
than  or  equal  to  384K  words  in  size.  It  requires  use  of  processor  and 
direct  channel  modifications  mentioned  above.  It  shall  be  capable  of 
supporting  one  or  two  real  processors.  (The  VMM  is  fundamentally  capable 
at  supporting  up  to  four  real  processors,  but  only  a single  processor  has 
been  tested  thus  far. ) 


As  a development  target,  the  VMM  shall  reduce  combined  system  throughput 
by  less  than  20  percent. 

Design  Overview  of  VMM 

The  VMM  at  its  most  gross  design  level  consists  of  two  parts:  start-up  and 
the  VMM  body. 

Start-up  is  executed  at  bootload  time  for  the  purpose  of  loading  and 
initializing  the  VMM.  Once  it  has  been  executed,  it  is  partially  overlaid 
and  not  used  again  until  the  next  VMM  bootload  process  is  initiated. 

The  VMM  body  consists  of  two  major  parts:  exception  processing  and 
dispatching.  Exception  processing  fields  all  faults  and  interrupts  (referred 
to  generically  as  exceptions)  and  initiates  appropriate  responses  to  them. 
Exceptions  may  cause  different  types  of  actions  to  take  place  depending 
upon  the  exception  which  occurred  and  the  environment  in  which  it  occurred. 

In  general,  exceptions  provide  the  method  used  by  VMM  to  maintain  the 
required  degree  of  control  over  the  system.  The  exact  way  in  which  this 
is  done  is  described  in  lower  level  documentation. 

After  exception  processing  is  complete,  the  dispatcher  is  invoked.  The 
dispatcher  gives  control  to  virtual  machines  at  the  appropriate  times  to 
maintain  the  proper  operation  of  the  virtual  machines  in  the  proper 
scheduling  sequence.  The  dispatcher  contains  the  scheduling  policy  of  the 
VMM  and  will  give  control  to  virtual  machines  in  accordance  with  this 
policy. 
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virtual  machines  (if  at  all),  the  dispatcher  will  return  control  in  different 
ways.  It  may  simply  return  to  the  last  point  of  execution  or  it  may  do  so  by 
simulating  a fault  or  interrupt. 

The  operating  systems  are  considered  to  be  subroutines  of  the  dispatcher 
In  this  design  overview.  The  dispatcher  will  give  control  to  one  or  the 
other  of  its  virtual  machines  at  appropriate  times.  Once  it  has  given 
control  to  an  operating  system,  the  dispatcher  will  have  finished  its  task 
and  will  not  again  be  invoked  for  execution  until  another  fault  or  interrupt 
has  been  processed  by  exception  processing.  The  fundamental  notion  of 
the  VMM  is  that,  aside  from  the  detailed  instructions  of  the  system,  the 
only  interfaces  that  exist  between  the  hardware  and  the  operating  system 
are  the  I/O  mailboxes  and  the  fault  and  interrupt  vectors.  In  most 
instances,  the  operating  system  can  be  allowed  to  execute  its  instructions 
and  those  of  its  users  at  full  speed  and  without  VMM  intervention  so  long  as 
their  addresses  are  constrained  to  lie  exclusively  within  the  domain  of  the 
operating  system.  Only  in  those  cases  where  fault  and  interrupt  vectors  or 
I/O  mailboxes  are  involved  is  it  necessary  for  the  VMM  to  gain  control  and 
perform  functions  which  will  prevent  operating  systems  from  interfering 
with  each  other  or  with  the  VMM.  Because  of  this  localization  of  control, 
the  VMM  can  control  its  guests  with  relative  efficiency  and  the  VMM  itself 
can  be  kept  to  a fairly  modest  size  and  complexity. 


The  CPU  modifications  used  in  support  of  VMM  allow  the  VMM  to  relocate 
the  address  space  in  which  each  operating  system  executes  so  that  each  one 
has  the  illusion  that  its  addresses  start  at  absolute  memory  address  zero 
and  proceed  upward  to  its  configured  maximum.  Given  this  illusion,  each 
operating  system  also  has  the  belief  that  it  owns  and  controls  the  real  fault 
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and  interrupt  vectors  of  the  system.  This,  of  course,  is  not  true.  The 
real  fault  and  interrupt  vectors  of  the  system  are  owned  and  controlled  by 
the  VMM. 

To  maintain  the  degree  of  control  which  is  needed  and  to  allow  the  operating 
systems  to  be  able  to  operate  upon  their  vectors  in  the  intended  manner,  the 
VMM  gains  control  on  every  fault  and  interrupt.  This  gives  the  VMM  the 
opportunity  to  keep  track  of  the  status  of  all  system  devices  which  it  must 
control  on  behalf  of  its  guest  operating  systems.  At  the  same  time,  it  gives 
the  VMM  the  opportunity  to  exercise  dispatching  control  over  its  guests. 

If  one  operating  system  has  used  a processor  for  its  fair  share  of  time,  the 
VMM  may  not  return  the  processor  to  it  in  the  event  of  a fault  or  interrupt. 

It  may  dispatch  the  processor  to  another  operating  system  instead.  Since 
the  VMM  intercepts  all  faults  and  interrupts  and  controls  the  dispatching  of 
the  processors  among  the  virtual  machines,  it  is  necessary  for  the  VMM  to 
be  capable  of  queueing  faults  and  interrupts  and  to  be  able  to  simulate  these 
in  a virtual  machine  at  the  time  a processor  is  dispatched  to  it. 

In  addition  to  gaining  control  on  every  fault  and  interrupt,  the  VMM  must 
gain  control  at  certain  other  key  times.  Chief  among  these  is  when  I/O 
is  about  to  be  initiated  (other  instances  are  elucidated  in  lower  level 
documentation).  This  is  essential  to  permit  the  VMM  to  perform  address 
translation  in  the  channel  programs  corresponding  to  the  relocation  of  the 
virtual  machine  within  the  real  address  space,  in  some  instances  to  perform 
device  and  channel  address  translation  and,  in  future  versions,  to  permit 
substitution  of  virtual  for  real  devices. 


At  the  general  conceptual  level,  the  controls  described  above  constitute  the 
heart  of  the  VMM.  More  complete  and  detailed  descriptions  are  contained 
in  lower  level  documentation. 

Test  Soecifications  for  VMM 


The  VMM  is  targeted  to  operate  with  an  overhead  of  less  than  20  percent. 
This  measure  shall  be  determined  by  executing  a certain  set  of  GCOS  jobs 
and  a MULTICS  script  as  separate  tasks  on  a freestanding  6180  system  and 
comparing  their  combined  execution  time  with  the  same  work  executing 
under  VMM.  In  performing  the  freestanding  measurements,  the  same 
system  resources  shall  be  used  as  are  used  when  executing  under 
the  VMM . 


Note  that  other  variations  on  the  method  of  measuring  performance  can  be 
used  and  are  perhaps  preferable. 

Known  Limitations  of  VMM 


The  dn355  shares  mass  storage  with  the  operating  system  which  it  is 
serving.  There  is  no  way  implemented  in  Version  1. 1 which  will  prevent 
the  355  from  accessing  or  writing  on  mass  storage  which  is  outside  of  its 
intended  area  of  use.  This  anomaly  will  be  repaired  in  future  versions  of 


FUNCTIONAL  DESCRIPTION  OF  EXCEPT-PROC 


The  exception  processor,  except -proc,  responds  to  all  faults  and  interrupts, 
safestores  the  processor  conditions  at  the  time  of  the  exception,  and  calls 
either  a fault  or  an  interrupt  handler  to  take  appropriate  action.  Except - 
proc  will  be  executed  by  any  processor  which  is  interruptable  at  the  time  an 
interrupt  occurs.  It  will  be  executed  by  the  processor  which  detects  a fault 
if  one  occurs. 

The  exception  processor  is  a key  element  of  VMM  because  it  gains  control 
of  the  system  at  those  points  and  times  when  it  is  essential  for  VMM  to  gain 
control  (for  example,  prior  to  the  execution  of  a cioc  instruction  in  a guest 
operating  system).  Of  course,  it  also  gains  control  in  some  instances 
which  are  of  no  interest  to  the  VMM,  such  as  when  a user  executes  a MME. 

In  the  latter  instances,  the  VMM  merely  returns  control  to  the  virtual 
machine  after  the  fault  is  processed. 

Design  Overview  of  Except-proc 

All  processors  share  a single  fault  vector  and  a single  interrupt  vector. 

The  vectors  contain  the  instruction  pair,  scu/tra.  In  order  for  the  results 
of  the  scus  to  be  associated  with  the  proper  processor  in  the  event  more 
than  one  processor  is  in  a single  vector  in  a short  time  interval,  ad  modifica- 
tion is  used  on  both  instructions  of  the  vector  pair.  This  causes  the  results 
of  each  safestore  to  be  placed  in  a separate  eight-word  area  and  each  transfer 
to  occur  to  a separate  location.  The  processor  number  of  each  processor 
is  set  in  its  switches  .and  this  value  is  stored  with  the  scu  information  so 
that  the  scu  results  can  be  correlated  with  the  stored  values  after  the 
vector  pair  has  been  executed. 
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A scu/tra  queue  is  used  to  allow  each  processor  to  safely  store  scu  and 
register  conditions. 


u 
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A trouble  fault  and  an  execute  fault  cause  the  system  to  abort;  all  other 
f/i's  store  the  scu  conditions  in  a scu  block  in  the  scu/tra  queue  and  then 
transfer  to  one  of  the  tra  blocks.  Six  lines  of  code  are  executed  that 
store  26  words  (pregs  (16),  regs  (8),  and  dsbr  (2))  in  that  same  tra  block,  and 
load  x4  with  the  address  of  the  tra  block.  The  last  instruction  in  every 
tra  block  is  a transfer  indirect  through  an  its  pair  to  the  fault -interrupt  - 
intercept -module  (film). 

Fiim  is  "pure"  code.  In  this  code  we  search  "back"  through  the  queue 
to  find  the  scu  conditions  stored  by  this  processor.  If  we  have  had  a fault 
in  the  fiim  we  may  find  more  than  one  scu  block;  we  continue  searching 
until  we  have  found  the  earliest,  or  the  one  that  originally  brought  us  to 
fiim. 

From  the  scu  and  tra  blocks  we  can  find  the  conditions  to  be  stored  in  a 
machine  conditions  block  for  use  by  f-proc  or  i-proc.  We  store  these 
machine  condition  blocks  in  the  vpdb  for  faults  or  interrupts  in  a vm,  in  the 
rpdb  for  faults  in  the  non -fiim  VMM  (e.  g.  absa),  and  we  do  not  store 
conditions  for  interrupts  in  the  VMM. 

After  storing  the  conditions  we  update  the  queue  to  show  that  these 
conditions  have  been  picked  up  and  stored.  We  also  verify  that  the  queue 
is  not  in  an  illegal  position. 

When  the  conditions  have  been  stored  and  the  queue  has  been  updated,  the 
processor  transfers  to  f-proc  or  i-proc  to  finish  processing  the  fault  or 
interrupt. 
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Exception  processing  consists  at  three  main  parts:  the  vectors,  the  scu/tra 
queue  and  film.  These  three  are  described  below  in  greater  detail. 


Fault/ Interrupt  Vector  Pairs --When  a fault  or  interrupt  occurs  the 
processor  enters  real,  absolute,  master,  6100  mode  and  transfers  to  an 
entry  in  the  fault/ interrupt  vector. 

The  vector  pairs  are  set  up  by  VMM  startup.  All  vector  pairs  except  for 
the  pair  corresponding  to  the  trouble  and  execute  faults  are  set  up  with  the 
same  two  instructions: 

scu  scu/tal,  ad 

tra  tra/tal,  ad 

Scu/tal  and  tra/tal  are  two  tally  words  in  low  core  that  reference  a location 
to  store  scu  conditions  and  a location  to  transfer  to,  respectively.  The 
trouble  fault  causes  the  system  to  abort.  Its  vector  pair  is  loaded  with 
these  two  instructions: 

scu  trb/flt/scu 

tra  trb/flt/tra 

At  trb/flt/tra  is  a transfer  to  the  system  abort  routine.  A similar  pair  of 
instructions  is  found  in  the  execute  fault  vector  pair. 

The  tra  tra/tal,  ad  causes  us  to  transfer  to  a set  of  code  (six  instructions) 
in  the  scu/tra  queue. 
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Scu/tra  Queue- -The  scu/tra  queue  is  composed  of  alternating  scu  and  tra 
blocks.  The  scu  blocks  are  eight  words  long  and  are  used  for  storing  the 
sen  conditions;  the  tra  blocks  are  32  words  long  and  are  used  to  store  the 
registers,  pointer  registers,  and  dsbr.  There  are  two  different  types  of 
tra  blocks,  since  some  of  the  storage  locations  in  the  tra  blocks  must  be  on 
mod  16  boundaries  and  40  is  not  a 0 mod  16  number. 

Type  i has  the  storage  area  for  pregs,  regs,  dsbr,  and 
six  words  of  code 

Type  2 has  the  storage  area  for  regs,  pregs,  dsbr,  and 
six  words  of  code 

The  six  lines  of  code  for  type  1 (2)  are 


sreg 

-10(26),  ic 

spri 

-27(19),  ic 

eax4 

-28.  ic 

sdbr 

- 5,  ic 

idbr 

vmm/dbr 

tra 

fiim/its/ptr,  * 

The  transfer  to  fiim/its/ptr,  * is  a transfer  to  the  film.  When  the  fiim  is 
entered  the  scu  and  register  conditions  have  either  been  stored  in  a valid 
location  or  they  have  not  and  the  fiim  will  detect  this. 

• If  the  stored  conditions  have  been  overwritten,  we  abort. 

• If  we  have  overwritten  the  end  of  the  queue,  we  abort. 

• If  we  have  written  into  the  " danger  zone"  of  the  queue,  we 
reset  the  tallies. 
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The  scu/tra/queue  management  is  explained  as  follows:  To  avoid  writing 


over  conditions  in  the  queue,  there  is  a variable  scu/ checks  for  each  real  ! 

processor  that  holds  the  address  of  the  scu  block  from  which  conditions  were 

last  stored  for  this  processor.  Lub/addr  (upper)  is  the  smallest  of  these 

addresses  and  we  try  to  never  write  past  it;  we  consider  the  information 

ahead  of  this  address  no  logger  important.  The  only  way  there  can  be  more 

than  one  scu/tra  block  pair  per  processor  is  if  the  tra  instruction  faults 

after  the  scu  instruction  is  executed,  or  if  there  is  a fault(s)  in  the  film. 

The  queue  is  set  up  with  the  following  pointers,  as  shown  below: 

start/q,  b/trsh,  q/trsh,  end/q,  lub/addr 

Every  time  the  scu/tal  is  incremented,  the  tra/tal  is  also  incremented.  If 
we  fault  during  the  scu  instruction,  we  get  a system  trouble  fault  and  abort 
the  system;  so  we  do  not  have  to  worry  about  unmatched  scu/tra  blocks 
and  we  know  that  the  information  stored  in  the  tra  block  is  always  stored 
after  (i.  e.,  more  recently)  than  in  the  scu  block. 


If  we  did  not  have  to  worry  about  updating  the  tally  words,  the  tra/tal  would 
always  point  to  the  tra  block  immediately  following  the  scu  block  that  scu/ 
tal  pointed  to.  However,  since  other  processors  can  be  using  the  tally  words 
while  one  processor  is  updating  them  in  the  fiim,  we  might  encounter  the 
situation  of  having  a processor  do  an  scu  scu/tal,  ad  just  before  the  updating 
of  the  scu  and  tra/tals  (which  are  updated  simultaneously),  so  that  the  tra 
tra/tal,  ad  instruction  executed  by  the  processor  causes  a transfer  to  the 
top  of  the  queue  while  its  scu  conditions  have  been  stored  at  the  bottom  of  the 
queue.  However,  the  tra/tal  address  is  still  circularly  greater  than  that 
of  the  scu/tal  so  we  use  the  address  in  tra/tal  to  check  if  we  have  over- 
written any  important  information. 
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We  define  the  lub  to  be  the  lowest  (number)  of  the  addresses  in  the  scu/ checks 
and  as  the  tallies  are  updated  to  the  top  of  the  queue  and  then  incremented,  we 
want  to  stop  before  this  address  in  the  lub.  To  help  insure  that  we  do  not 
store  past  this  address,  we  define  b/trsh  to  be  b/trsh/upd  words  before 
(circularly)  the  lub  and  try  to  stop  at  this  address.  We  use  the  address 
in  tra/tal  to  check  for  b/trsh. 

If  we  find  that  the  tra/tal  address  is  within  b/trsh/upd  words  of  the  lub,  we 
abort.  In  the  same  manner,  we  attempt  to  avoid  writing  past  the  end  of  the 
queue  by  defining  a q/trsh  to  be  a fixed  distance  before  the  end  of  the  queue 
and  we  update  the  tallies  whenever  the  address  in  tra/tal  is  greater  than 
this  value  in  q/trsh. 

Below  are  the  six  possible  arrangements  of  the  lub,  b/trsh,  and  tra/tal: 


b / trsh 


b/trsh 


b/trsh 


b/trsh 


b/trsh 


b/trsh 


Conditions  1,  4 and  6 should  cause  an  abort. 
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Film- -The  last  instruction  is  the  tra  block,  a transfer  indirect  through  an 
its  pair,-  causes  us  to  enter  append  mode.  As  its  name  implies,  fiim  is 
pure,  i.e.,  it  can  be  executed  simultaneously  by  several  processors.  The 
outline  below  describes  the  operation  of  fiim. 


A.  INITIALIZE 

1.  The  first  ten  instructions  in  fiim  are  inhibited.  As  soon  as 
fiim  is  entered  pr.  rsdb  and  pr.lcdb  are  set  to  correctly 
reference  the  real  system  data  base  (rsdb)  and  low  core  data 
base  (lcdb)  respectively. 

2.  Then  rsdb  |inhib/scu/ mask  is  used  to  set  the  memory  controller 
mask  (smcm)  in  the  scu  with  lower  order  memory.  Once  this 
is  done  it  is  no  longer  necessary  for  the  instructions  to  be 
inhibited. 

3.  Next,  the  processor  switches  are  read  to  determine  which  real 
processor  is  executing  the  fiim.  With  this  information  we  can 
load  br.  rpdb  with  a sdw  number  that  corresponds  to  an  unique 
segment  for  each  real  processor  known  as  the  real  processor 
data  base  (rpdb).  While  executing  the  code  that  accomplishes 
this  we  set  x7  to  the  processor  number.  There  is  an  array, 
temp,  with  a word  for  every  read,  processor.  Thus  we  can 
use  "temp,  7"  as  a temporary  storage  location  and  the  code 

is  still  pure. 
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4.  As  soon  as  the  rpdb  has  been  determined  the  clock  of  scuO  is 
read  and  the  value  stored  in  rpdb  vmm  entry  time. 

5.  Next  x5  is  loaded  with  lcdb|  scu/tal.  The  conditions  for  the 
real  processor  were  stored  at  least  40  words  before  this 
value,  depending  on  how  many  processors  have  since  executed 
a fault/interrupt  vector  pair. 

B.  FIND  A VALID  scu  BLOCK  FOR  THIS  PROCESSOR 


In  most  cases  there  will  only  be  one  set  (values  stored  in  the  scu 
and  tra  blocks  together)  of  conditions  stored  for  the  real  processor 
executing  fiim  (see  the  above  discussion  of  how  the  scu/tra  queue 
works).  However,  there  may  be  two  or  more  sets  of  conditions 
stored,  and  we  want  to  find  the  first,  or  earliest,  stored  set  in 
order  to  save  the  conditions  with  which  fiim  was  entered.  Since 
x4  contains  the  address  of  the  tra  block,  we  are  searching  only  for 
the  earliest  (oldest)  scu  block  for  this  processor. 

To  best  utilize  the  queue  we  want  scu/ checks  for  this  processor  to  be 
set  with  the  address  of  the  most  recent  scu  block  found  for  this 
processor,  so  we  use  rpdb/ first -time  ind  as  a flag;  when  it  is 
nonzero  we  have  not  yet  found  a valid  scu  block  for  this  processor; 
when  it  equals  zero  we  have  found  a valid  scu  block  and  we  have 
saved  its  address  in  rpdb|  saved/ tally /addr. 


Find  an  scu  Block  for  this  Processor 


As  stated  above,  x5  is  set  equal  to  lcdb  | scu/tal  so  x5  either 
equals  a valid  scu  block  address  or  it  is  > q/trsh. 

We  search  backwards  through  the  queue  looking  for  a s<  1 
block  stored  by  this  processor.  Word  3 of  the  scu  data  contains 
the  processor  number  in  bits  28-29.  If  this  is  equal  to  x7  we 
check  to  see  if  we  have  reached  the  address  of  the  scu  block 
used  for  this  processor  the  last  time  it  was  in  fiim.  This 
address  is  kept  in  rsdb  |scu/ checks,  6.  If  we  have  not 
reached  it  yet  we  continue  backing  up  the  queue;  if  we  reach 
rsdb  | scu/ checks,  7 we  decide  that  there  was  no  block  of 
conditions  stored  for  this  processor  and  we  abort,  abort/ 6. 

See  if  Address  for  this  scu  Block  is  Valid 

x5  is  the  address  of  a scu  block  stored  by  this  processor.  We 
check  bit  18  or  the  third  scu  word  to  see  if  the  conditions  have 
already  been  stored.  Unless  we  are  in  the  "possible  garbage 
area"  (x5  > q/trsh)  we  abort  if  the  conditions  are  already 
stored,  abort/8.  This  bit  (18)  is  set  near  the  end  of  the  fiim 
and  is  reset  when  new  scu -conditions  are  stored  in  this  block 
via  the  scu  instruction  in  the  fault/interrupt  vector  pair. 


If  the  conditions  have  not  yet  been  picked  up  and  stored,  we 
proceed  under  the  assumption  that  either  (i)  the  scu  conditions 
are  valid,  (ii)  they  were  stored  as  a result  of  a fault  in  the 
fiim  and  we  will  continue  looking  for  the  earliest  block  for  this 
processor,  or,  (iii)  something  is  wrong  with  the  queue  and  after 
discovering  this  later  in  the  fiim  we  will  abort. 

As  may -be -valid -conditions  we  set  up  pr.vmdb  and  pr.  vpdb  to 
reference  the  virtual  machine  data  base  (vmdb)  and  virtual 
processor  data  base  (vpdb)  respectively.  Then  we  store  the 
second  word  of  the  scu  conditions  into  rpdb/scu/fault/word 
so  that  we  can  pick  it  up  before  leaving  fiim.  We  cannot  wait 
until  then  to  look  for  this  second  word  because  x5  will  no 
longer  reference  the  scu  block  and  pr5  will  not  be  valid  pointer 
to  stored  conditions  in  the  case  of  an  interrupt  in  the  VMM. 

C.  STORE  CONDITIONS 


1.  Determine  Type  of  Conditions- -Fault  or  Interrupt 

From  word  2 or  the  scu  block,  lcdb/scu/fault/ data/ word,  5,  we 
can  check  bit  35  to  see  if  we  had  an  interrupt  or  a fault;  then 
from  the  real/ virtual  indicator,  bit  3 2 of  the  fifth  word  of  the 
scu  conditions,  we  can  discover  if  this  fault /interrupt  occurred 
in  the  VMM  or  the  VM. 

If  there  was  a fault  in  the  fiim. we  return  to  look/ for/ scu  to  find 
another  scu  block  for  this  processor. 

I 
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If  there  was  a fault  in  a VM  we  set  bit  0 of  rpdb|  ind/word  to  show 
that  fiim  was  entered  from  a VM  by  a fault. 

If  there  was  an  interrupt  we  set  bit  6 in  rpdb|  ind|  word. 

If  the  interrupt  was  in  the  VM  we  zero  bit  0 of  rpdb|  ind/word 
to  show  that  fiim  was  entered  from  a VM  by  an  interrupt. 


We  pick  up  conditions  from  lcdb/0,  5,  where  x5  is  the  address 
of  an  scu  block,  and  store  them  in  5/ 0,  where  pointer  register 
5 -pr5-  has  been  set  up  to  reference  the  first  word  of  a 48  word 
block  of  machine  conditions.  pr5  is  set  as  follows: 

Fault/ Interrupt  in  the  VMM --For  a fault  in  non-fiim  VMM, 
pr5  points  to  rpdb|  vmm/cond/ start,  3.  vmm/cond/ start  is 
the  first  address  of  a list  of  machine  conditions  blocks,  and  x3 
determines  which  block  we  will  store  into. 

For  an  interrupt  in  the  dispatch  segment,  no  conditions  are 
stored  and  pr5  has  no  meaning  in  this  instance. 

Fault/ Interrupt  in  the  VM--For  a fault  or  an  interrupt,  pr5 
references  vpdb | vp/ mach/ conditions. 
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Store  the  Conditions 

Pointer  and  lengths  are  stored  at  all  times. 

For  the  mlr  instructions  we  have  to  make  sure  that  we  work 
with  character  addresses;  in  the  case  of  desc9a  these  character 
addresses  are  four  times  the  word  addresses.  We  store  the 
scu  conditions  and  the  value  of  the  clock  that  we  read  on 
entering  fiim  and  then  saved  in  rpdb|  vmm/ entry/ time. 

From  x4,  the  word  address  of  the  tra  block,  we  can  determine 
if  we  stored  conditions  in  a typel  of  a type 2 block.  From  each 
block  we  pick  up  and  store  the  pregs,  regs,  and  the  dsbr. 

A 'scpr'  with  a tag  of  01  stores  the  fault  register  in  5/mc / 
fault/ reg. 

4.  After  all  conditions  have  been  stored,  the  stored  bit  is  set  in 
the  scu  conditions  in  the  queue. 

D.  QUEUE  UPDATE 

In  the  queue  updating  code  we  update  scu/ checks,  the  lub,  b/trsh, 
etc.  if  necessary  while  checking  for  possible  error  conditions. 

On  reaching  queue -update  we  check  the  stored  bit  of  the  lub.  If 
it  is  one  we  proceed,  but  if  it  0 the  l.ub  has  been  overwritten  and  we 
abort  since- we  can  no  longer  be  confident  that  the  conditions  we  picked 
up  from  the  scu/tra  queue  are  valid,  (abort/ 18) 
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While  updating  the  queue  we  lock  the  code  with  rsdb|fiim/q/lock. 

1.  Update  scu/ checks 

We  store  the  address  of  the  most  recent  scu  block  found  for 
this  processor  (this  address  has  been  saved  in  rpdb/saved/ 
tally/addr)  in  scu/ checks.  7. 

2.  Update  lub,  b/trsh,  etc,  if  necessary 

If  this  processor  owns  the  lub  we  have  to  search  all  the  scu/ 
checks  of  the  active  processors  to  find  the  lowest  address. 

We  put  this  address  into  the  lub,  i.  e.,  into  rsdb|lub/addr 
(upper  half)  and  calculate  a new  rsdb|b/trsh  by  circularly 
subtracting  rsdb|b/trsh/upd  words.  If  the  lub  is  less  than 
b/trsh/upd  words  from  the  start  of  the  queue  (rsdb/start/q) 
we  wrap  around  to  the  bottom  of  the  valid  queue,  above  rsdb| 
q/trsh,  and  continue  moving  backwards  until  we  have  gone 
b/trsh/upd  words. 

3.  Check  Queue  Position 

There  are  six  possible  queue  positions  with  respect  to  the  lub, 
tra/tal,  and  b/trsh.  We  check  for  the  three  acceptable  positions 
and  abort  for  the  other  three,  wait/abort.  See  the  information 
on  the  scu/tra  queue  above  for  a discussion  of  which  positions 
should  cause  aborts. 


. • — 
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E.  TRACE 

If  rsdb | trace/flag  * 0 we  do  not  perform  a trace.  If  it  is  not 
equal  to  0 we  enter  trace  by  one  of  two  entries: 

Entry  point  1 (trace/ 1)  is  used  when  we  wish  to  save  the  trace  code, 
words  2,  5,  and  7 of  vp/mach/ conditions,  rpdb|  vm/vp/no,  ind/word, 
vpdb|vp/ state,  and  the  real  proc,  number. 

Entry  point  2 (trace/ 2)  is  used  when  we  wish  to  save  the  eight 
words  we  have  stored  into  rpdb|  trace /conditions. 

Trace/ 1 and  trace/ 2 are  both  entered  with  a tsx6,  x5,  x7,  and  pr6 
are  changed  for  both  entry  points;  rpdbj trace /conditions  are  changed 
if  we  enter  at  trace/ 1. 

F.  LEAVE  fiim 

The  Q register  is  loaded  with  the  fault/interrupt  number,  right 
justified. 

Rpdb/interrupt/flag  is  used  to  tell  whether  it  was  an  interrupt  or 
a fault  that  brought  us  to  fiim.  If  interrupt/flag  * 0 it  was  a 
fault  and  we  transfer  to  f-proc  through  an  its  pair.  If  interrupt /flag 
does  not  equal  zero  we  transfer  to  i-proc  through  an  its  pair  to 
handle  the  interrupt.  In  both  instances  we  enter  the  second  word 
of  the  particular  segment  since  the  first  word  is  an  error  trap. 


Functional  Description  of  I-proc 


I-proc  is  the  VMM  module  which  handles  the  processing  of  all  VMM  and  VM 
interrupts.  I-proc  is  invoked  by  the  fault /interrupt  interceptor  module 
after  the  occurrence  of  an  interrupt.  Upon  entry  to  this  module  the  state 
of  the  processor  has  been  saved  for  those  interrupts  which  occurred  while 
the  processor  was  in  virtual  mode*  If  the  interrupt  happened  in  real  mode 
(i.  e. , while  in  the  VMM)  then  the  state  of  the  processor  will  not  have  been 
saved.  The  reason  the  processor  state  is  not  preserved  is  that  while  in 
real  mode  the  processor  is  inhibited  from  receiving  interrupts  except 
during  one  instance  while  in  the  VMM  dispatching  module.  If  an  interrupt 
occurs  at  this  point,  it  is  not  necessary  to  save  the  processor  state. 
Interrupts  cannot  occur  at  any  other  instance  because  the  processor  is 
inhibited  through  the  use  of  the  inhibit  bit  and  the  SCU  masks. 

The  processing  performed  by  I-proc  and  its  related  routines  entails  a 
simulation  of  the  functions  performed  by  the  I/O  multiplexor  (iom)  and  the 
system  controller  (scu).  For  this  simulation  I-proc  must  appropriately 
update  the  virtual  address  space  corresponding  to  the  virtual  machine 
associated  with  the  interrupt  being  processed.  This  updating  consists  of 

modifying  the  virtual  mailboxes,  interrupt  multiplex  words,  and  status 
words. 

Usage  Information  for  I-proc --This  module  relies  completely  on  the  data- 
bases constructed  for  the  real  and  virtual  environments.  When  the  interrupt 
processor  is  invoked  by  fiim,  it  is  expected  that  the  interrupt  type  (int- 
level)  will  be  contained  in  the  lower  portion  cf  the  Q register.  Upon  exit 
from  I-proc,  the  appropriate  virtual  addresses  and  VMM  data  bases  will 
have  been  updated  to  reflect  the  completion  of  the  interrupt  processing. 
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Resource  Objectives  of  I-proc--The  objective  of  the  interrupt  handler  is  to 
efficiently  simulate  for  a virtual  machine  the  operation  of  the  peripheral 
subsystems  in  regard  to  the  termination  operations  performed  in  processing 
on  I/O  request. 

Design  Overview  of  I-proc  - -At  this  level  of  the  interrupt  processor  the 
current  time  of  day  is  read  from  the  scu  with  memory  address  0.  This  is 
used  to  meter  the  time  spent  processing  interrupts.  Then  according  to  the 
type  of  interrupt  being  processed  as  determined  by  the  interrupt  level 
number,  the  module  corresponding  to  I/O  system  first  initiating  the 
interrupt  will  be  invoked.  All  processing  of  the  interrupt  will  be  performed 
at  these  lower  levels  with  the  aid  of  a set  of  common  subroutines. 

Functional  Description  of  iom/int  --The  processing  of  all  interrupts  from  an 
iom  are  handled  by  iom/int.  The  amount  or  extent  to  which  the  interrupt 
is  processed  and  simulated  for  the  corresponding  virtual  machine  is 
dependent  upon  the  interrupt  type. 

At  this  level,  the  interrupt  multiplex  word,  imw,  corresponding  to  the 
interrupt  received  is  examined  to  determine  which  if  any  channels  require 
interrupt  processing.  Each  channel  interrupt  is  processed  independently. 

If  the  interrupt  is  from  the  overhead  channel  six  then  processing  will  be 
performed  by  the  psia-proc  module.  All  other  interrupts  from  overhead 
channels  will  cause  a VMM  abort. 

The  processing  of  payload  channel  interrupts  is  handled  separately.  Non- 
terminate  interrupts  will  be  handled  by  the  module  non/ term /int.  If 
the  payload  interrupt  is  the  result  of  an  interrupt  from  a 355,  then  the 
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module  DN355  will  be  invoked.  The  only  interrupt  not  covered  by  the  above 
two  cases  is  the  terminate  interrupt.  The  terminate  interrupt  processing 
consists  of  determining  the  real-to-virtual  map,  updating  the  virtual  mail- 
box entries,  adjusting  and  storing  the  associated  status  word,  and  setting 
the  proper  virtual imw  bit.  After  this  processing  has  been  performed  the 
data  storage  for  this  request  is  released  and  the  processing  of  the  next  I/O 
request  for  the  real  channel  or  any  of  its  crossbars  is  initiated  by  iom-proc. 

Usage  Information  for  iom/int--This  submodule  is  called  from  within 
the  interrupt  processor,  I-proc,  to  process  any  iom  related  interrupts. 

Upon  entry  the  number  of  the  real  iom  causing  the  interrupt  is  known  and 
the  type  of  interrupt  is  provided  by  the  imw  level.  The  contents  of  the  imw 
services  to  drive  this  module  such  that  all  channel  interrupts  are  processed. 
Upon  termination  of  this  routine  the  interrupt  has  been  simulated  in  virtual 
space  and  the  appropriate  interrupt  cell  in  the  virtual  machine's  scu  data- 
base has  been  set.  Also,  a connect  has  been  performed  for  the  next  queued 
I/O  entry  for  each  channel  processed. 

Functional  Description  of  non/ term/ int- -This  routine  is  used  to  process 
interrupts  which  are  not  expected  by  the  VMM  (i.  e. , non-terminate  interrupts) 
with  the  exception  of  channel  six  interrupts.  Since  they  are  not  expected, 
the  virtual  mappings  are  not  determined  as  in  the  terminate  case.  Instead, 
the  mapping  is  calculated  from  a set  of  mapping  constructed  at  VM  start- 
up time.  Once  the  virtual  map  is  determined,  the  VM's  corresponding  imw 
and  SCW  interrupt  cells  are  set  to  complete  the  processing  of  this  interrupt. 
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Known  Limitations  of  non/ term /int- -This  module  assumes  only  dedicated 
peripherals  are  configured  to  a VM.  In  addition,  only  special  interrupts 
and  interrupts  from  a 355  will  be  processed.  If  a marker  or  initiate 
interrupt  is  received  it  will  result  in  a VMM  abort. 

Functional  Description  of  real/ virtual/ map- -This  module  transforms  the 
mapping  of  the  real  device  currently  being  processed  into  a mapping  of  the 
associated  virtual  device.  The  virtual  machine,  iom,  channel,  and  device 
numbers  are  determined  as  well  as  descriptors  describing  the  VM's  data 
base  within  rsdb  and  the  VM’s  address  space. 

Usage  Information  for  real/ virtual/ map- -The  input  variable  vir-map 
(qu)  is  used  to  create  the  variables  vir-iom(xl),  vir-ch(x3),  vmpr(pr6)  and 
vmdbtr(pr7).  After  the  module  is  finished  executing,  control  will  be 
returned  to  the  x6  within  the  current  segment. 

Functional  Description  of  update/ virtual/ sew- -This  module  is  responsible 
for  updating  the  virtual  status  control  word(scw)  corresponding  to  the  virtual 
channel  whose  interrupt  is  currently  being  processed.  This  updating  shall 
be  identical  to  that  performed  by  the  iom-B  on  a status  service.  The  iom-B 
allows  three  types  of  status  queues  - a list,  a circular  queue  with  four 
entries  and  a circular  queue  with  16  entries.  The  type  of  queue  as  well  as 
the  next  entry  in  the  queue  is  specified  by  the  sew.  A tally  field  is  also 
provided  to  determine  the  length  of  a status  list  when  the  list  option  is 
employed. 

Usage  Information  for  update/virtiial/scw--In  order  to  process  the 
virtual  sew,  this  module  must  be  called  with  the  following  variables: 
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1)  vir/iom 

2)  vir/ch 

3)  rpdb 

4)  vmpr 

5)  vmdbtr 

The  above  variables  will  be  preserved  during  the  module' s execution.  In 
addition,  the  virtual  sew  corresponding  to  the  interrupt  to  be  processed 
will  be  updated  and  vir/sew/address  will  be  the  virtual  address  used  to 
store  the  status. 

W 

Design  Overview  of  update/ virtual/scw- -The  function  of  update- virtual  - 
sew  is  to  perform  the  same  type  of  servicing  for  the  virtual  sew  as  perform- 
ed by  the  iom  for  a real  sew.  This  includes  updating  the  sew  pointer  and 
decrementing  the  sew  tally.  The  exact  procedure  for  this  sew  modifica- 
tion is  presented  in  detail  in  the  iom  EPS-1. 

In  addition  to  the  update  function,  the  routine  also  tests  the  store  address 
for  the  next  status  pair  to  insure  that  the  status  will  be  stored  within  the 
proper  virtual  machine's  address  space.  This  address  is  used  by  update/ 
virtual/ status  to  store  the  virtual  status. 

Functional  Description  of  update /virtual/  status  --The  function  of  update/ 
virtual/ status  is  to  perform  the  updating  of  the  virtual  mailbox  lpw  and  lpwe 
and  the  virtual  status  pair.  The  virtual  mailbox  sew  is  updated  by  the  module 
update/ virtual/scw. 
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Both  the  lpw  and  lpwe  are  modified  by  the  iom  in  the  process  of  executing 
a channel  program.  When  an  interrupt  is  received  from  this  channel,  the 
state  of  these  two  mailbox  words  may  reflect  the  presence  of  the  VMM. 
Therefore,  the  address  fields  and  lpw  state  bits  (AE,  relative,  restricted) 
must  be  virtualized. 

The  status  pair  is  similar  to  the  lpw  and  lpwe  in  that  it  must  also  be 
virtualized.  The  first  word  of  the  status  pair  contains  an  address  extension 
field  which  must  be  corrected  to  reflect  the  virtual  address  extension.  The 
dew  residue  word  (the  second  word  of  the  status  pair)  must  also  be 
virtualized  to  reflect  a possible  VM  address  space  offset  from  a 256K 
memory  boundary. 

Usage  Information  for  update/virtual/status--The  following  variables 
must  be  provided  to  update/ virtual/ sew  upon  entry: 


1) 

ledb 

6) 

vir/ch 

*2) 

rpdb 

7) 

vmpr 

3) 

real  / iom 

8) 

vmdbtr 

4) 

real/  chan 

9) 

vir/sew/addr 

5) 

vir/iom 

These  variables  will  not  be  modified  during  the  execution  of  this  module. 
However,  this  module  does  update  the  virtual  channel's  lpw,  lpwe  and 
status  pair  within  the  VM's  address  space. 
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Resource  Objectives  of  update/ virtual/  status  --The  updating  of  the  lpw, 
lpwe,  and  status  pair  adds  greatly  to  the  overhead  involved  with  processing 
an  interrupt.  This  is  especially  true  when  the  virtual  machine  ignores  the 
virtualization  in  most  cases. 

Known  Limitations  of  update /virtual/ status  --In  order  to  keep  VMM  over 
head  at  a minimum,  a complete  virtualization  of  the  VM's  channel  mailbox 
is  not  always  performed.  Only  in  the  event  of  status  pair  w;ith  non-zero 
major  or  minor  status  fields  will  the  VM's  lpw  and  lpwe  be  updated.  It  is 
assumed  that  the  lpw  and  lpwe  will  not  be  examined  when  the  major/minor 
status  fields  are  zero. 

ZHngti°nal  Description  of  set/vir/imw--Qne  of  the  functions  of  set/vir/imw 
is  to  simulate  the  scu  in  setting  its  interrupt  cells.  Within  the  system 
control  unit  (scu)  there  exist  32  interrupt  cells.  Each  of  these  cells 
corresponds  to  one  of  the  32  interrupt  types.  The  proper  cell  is  set  when 
the  scu  receives  a set  interrupt  cell  command  from  one  of  the  subsystems 
configured  to  the  scu.  These  cells  together  with  the  scu's  interrupt  can  be 
delivered  to  a configured  processor.  This  same  function  is  simulated  by 
set/vir/imw  by  setting  the  corresponding  bit  in  the  virtual  machine's  data 
base  word  representing  the  scu  interrupt  cell. 

Another  of  the  functions  performed  by  set/vir/imw  is  to  set  the  proper  imw 
word  in  the  virtual  machine's  address  space.  When  the  iom  sends  the  scu 
a set  interrupt  cell  command,  it  also  requests  the  scu  to  set  a bit  in  the 
imw  word  corresponding  to  the  interrupt  cell  just  set.  The  bit  set  within 
the  imw  word  represents  the  channel  on  the  iom  which  initiated  the  interrupt. 
This  iom/ scu  function  is  simulated  by  setting  the  proper  bit  in  the  virtual 
machine's  imw. 
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The  final  task  performed  by  the  set/vir/imw  is  to  construct  a trace  entry. 
When  the  VMM  system  trace  indicator  is  true,  a trace  entry  consisting  of 
the  real  status  pair,  the  real  channel  index,  and  the  interrupt  type  is  built 
and  the  system  trace  module  is  called. 

Usage  Information  for  set/vir/imw --This  module  requires  the  variables 
vmpr  (pr6)  and  vmdbtr  (pr7)  be  specified  in  the  described  hardware 
registers. 

Functional  Description  of  dn355/int--The  purpose  of  this  module  is  to 
process  interrupts  from  355s  which  are  connected  through  the  modified 
DIA  to  the  iom.  For  VMM  step  1.  1 development  only,  dedicated  355s  will 
be  allowed.  A dedicated  355  is  one  which  is  only  configured  virtually  to 
one  virtual  machine. 

Under  the  dedicated  355,  the  355  mailbox  directly  will  be  accessible  to 
the  355  and  may  be  directly  updated  without  VMM  intervention.  Therefore, 
355  interrupt  processing  will  not  require  a simulate  of  the  operations 
performed  by  the  355.  However,  the  scu  functions  must  still  be  simulated 
for  the  corresponding  virtual  machine.  This  involves  the  setting  of  the 
appropriate  interrupt  cell  in  the  VM's  virtual  scu's  data  base  and  the  up- 
dating of  the  corresponding  virtual  imw  within  the  VM's  address  space. 

Usage  Information  for  dn355/int--This  module  is  called  from  the  iom 
interrupt  handler  when  an  interrupt  from  a 355  is  detected.  When  called 
the  interrupt  is  processed  as  explained  earlier.  Upon  entry  to  the  module  the 
355  channel  index  is  known  (x5)  as  well  as  the  address  of  the  channel's 
device  descriptor  table  (al).  Most  of  the  processing  of  the  355  interrupt  is 
performed  by  the  two  modules:  real/ virtual/ map  and  set/vir/imw. 


Design  Overview  of  dn355/int--The  functionality  of  this  module  is 
simplified  by  the  use  of  the  two  modules:  real/ virtual/ map  ana  set/vir/imw. 
Upon  entry  to  dn355/int,  the  real  to  virtual  map  for  the  dedicated  355  is 
obtained  from  the  first  entry  in  the  channel's  device  descriptor  table.  The 
real/ virtual /map  routine  is  then  called  to  determine  the  virtual  machine 
number , virtual  iom  number,  and  the  virtual  channel  index.  This  later 
module  also  sets  up  the  pointer  registers  for  the  VM's  data  base  and  virtual 
address  space.  Upon  return  the  module  set/vir/imw  is  called  to  set  the 
VM's  corresponding  imw  and  to  set  the  proper  interrupt  cell  in  the  virtual 
scu  data  base  within  the  VMM.  At  this  point  the  processing  of  the  355 
interrupt  is  complete  and  control  is  returned  to  iom/int  to  process  the 
next  channel  interrupt. 


Functional  Description  of  vmm/int--This  routine  processes  the  VMM  soft- 
ware interrupts.  The  four  VMM  software  interrupts,  one  for  each  of 
four  possible  real  processors  on  a VMM  system,  are  assigned  to  interrupt 
cells  3,  8,  9,  and  10  respectively.  These  software  interrupts  are  set  by  a 
smic  instruction  in  dispatch  when  there  exists  an  outstanding  connect  fault 
or  interrupt  for  the  vm/vp  currently  executing  in  that  real  processor.  The 
use  of  these  interrupts  allow  the  virtual  processor  to  be  interrupted  when 
the  vp  reaches  an  interruptable  state. 

Design  Overview  of  vmm/int--The  processing  of  a VMM  software 
interrupt  simply  involves  the  resetting  of  the  flag:  proc/spec/int/flag. 

This  is  used  to  indicate  that  the  VMM  software  interrupt  cell  for  this  real 
processor  is  not  set. 


A- 31 


Functional  Description  of  f-proc 


r 


The  purpose  of  the  fault  processor  is  to  determine  the  reason  for  a hardware 
fault  and  process  it  accordingly.  For  metering  purposes,  the  fault 
processor  will  keep  statistics  concerning  the  time  taken  to  process  faults. 
There  are  two  major  categories  of  faults  distinguished  by  the  fault  processor: 

1)  VMM  faults --These  are  faults  that  occur  within  the  VMM  and  are, 
therefore,  up  to  the  VMM  to  process  for  itself. 

2)  VM  faults --These  are  faults  that  occur  while  a virtual 
processor  "has  control"  of  a real  processor.  These  faults 
range  in  processing  by  the  VMM  from  completely  to  not  at  all. 

There  are  36  different  "hardware"  faults  for  each  of  the  above  mentioned 
categories,  making  a total  of  72  individual  fault  processors.  These 
individual  processors  are  described  in  lower  levels  of  documentation. 


Resource  Objective  of  f-proc- -The  fault  processor  should  be  time  optimized 
as  much  as  possible  to  the  more  common  functions.  The  least  common 
functions  need  not  be  so  optimized. 


Design  Overview  of  f-proc- -The  time  the  fault  processor  was  entered  shall 
be  saved  for  metering  purposes.  Metering  shall  be  accomplished  at  the 
end  of  fault  processing  and  is  dependent  of  the  metering  routine  defined  in 
a lower  level  of  documentation.  The  input  parameter,  the  fault  number 
causing  fault  processor  entry,  together  with  the  variable  holding  the  count 
of  VMM  faults  not  yet  processed  determines  the  fault  processing  to  be 
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attempted.  If  there  is  a VMM  fault  outstanding,  then  the  input  parameter 
will  be  assumed  to  give  the  number  of  the  last  fault  (not  yet  processed)  that 
occurred  within  the  VMM.  The  fault  processor  will  always  handle  faults 
occurring  within  the  VMM  before  it  handles  those  occurring  within  a VM. 

Once  the  fault  processor  determines  where  the  fault  occurred,  it  determines 
what  fault  handler  to  select  (depending  on  the  input  parameter).  The 
appropriate  fault  handler  is  then  invoked.  Metering  is  done  on  return  from 
the  particular  fault  handler. 

Temporarily  all  faults  occurring  within  the  VMM  will  cause  the  system  to 
abort. 

Variables  for  f-proc--All  variables  are  defined  in  their  respective  include 
files.  All  variables  referenced  at  this  level  of  abstraction  are  in  the  rpdb 
include  file. 

Known  Limitations  of  f-proc--No  provision  for  the  handling  of  faults 
occurring  within  the  VMM  has  yet  been  made,  except  that  the  system  will 
abort.  Metering  measurements  are  slightly  biased  toward  the  low  side  due 
to  the  time  of  execution  of  the  metering  instructions  themselves.  Also,  when 
getting  to  processing  VMM  faults,  the  entry  time  to  the  fault  processor  may 
be  overwritten  before  a measurement  is  actually  made  (thus  biasing  to  the 


low  side). 


Functional  Description  of  vp/ipr 


If  the  illegal  slave  bit  in  the  scu  conditions  is  not  on,  then  the  fault  is 
returned  as  is.  The  virtual  fault  register  is  updated  appropriately. 

If  the  illegal  slave  bit  is  on,  then  the  modes  are  checked  to  see  whether  the 
instruction  was  executed  in: 

1)  6100,  absolute,  master  modes 

2)  6100,  append,  master  and  privileged  (PPR.  P*  1) 

3)  6000  and  master  modes 


If  the  instruction  fault  was  executed  in  6000  and  slave  modes,  then  a table 
lookup  on  the  faulting  op  code  is  performed.  If  the  op  code  is  in  the  table, 
then  x3  will  contain  the  address  plus  one  of  the  locations  associated  with 
the  "faulted  on"  op  code.  The  VMM  special  command  processor 
(VMM/cwd/proc)  will  handle  the  next  step  in  instruction  simulation.  If  the 
op  code  is  not  in  the  table  or  the  instruction  was  not  attempted  in  any  of  the 
above  mentioned  modes,  then  an  illegal-in-slave  fault  must  be  returned. 

For  MULTICS,  illegal -in -slave  processing  requires  only  that  the  virtual 
fault  register  be  updated.  For  GCOS,  illegal-in -slave  processing  requires 
that  the  fault  number  be  changed  to  a command  fault  (in  the  sw  conditions) 
as  well  as  that  the  virtual  fault  register  be  updated. 


Functional  Description  of  vp/acv 


The  access -violation  fault  is  one  of  the  6100  faults  that  may  need  to  b« 
returned  to  a 6000  virtual  processor  (under  a different  fault  name,  of 
course).  The  only  access  violation  type  that  will  be  returned  to  a 6000  vp  is 
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the  out-of-segment -bounds  fault.  This  fault  will  occur  (on  a 6000  vp)  only 
if  memory  is  mapped  via  a segment  descriptor  word.  If  an  out-of-segment- 
bounds  fault  is  detected  for  6000  vp,  then  a store  fault  (non-existent-address) 
is  simulated  for  the  vp.  An  access  violation  occurring  within  a 6100  vp  will 
be  given  back  to  the  vp  as  is. 

Design  Overview  of  vp/acv--If  the  access  violation  occurs  while  a 6100  vp  has 
control  of  the  processor  (vmdls/vm/type/0),  then  the  fault  will  be  returned 
to  the  vp  as  is.  If  the  access  violation  (acv)  occurs  while  a 6000  vp  is 
in  control,  then  two  cases  are  relevant: 

1.  If  the  acv  is  an  out-of-segment-bounds,  then  the  6000's 
memory  is  assumed  to  be  bounded  by  a segment  descriptor 
word.  A store  fault  (non-existent  address  is  returned). 

2.  If  the  acv  is  not  an  out -of -segment -bounds,  then  the  VMM  is 
expected  to  have  set  up  the  decor  incorrectly  and  the  VM  is 
aborted. 

If  a store  fault  is  to  be  returned: 

1.  The  out-of-segment-bounds  bit  in  the  scu  conditions  is 
zeroed  out. 

2.  The  non-existent  address  bit  in  the  scu  condition  is  set  on. 

3.  The  fault  field  (containing  access  violation  number)  is  replaced 
by  the  store  fault  number. 
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4.  The  virtual  fault  register's  non-existent  address  bit  is  set  on. 

Known  Limitations  of  vp/acv--If  GCOS  is  guaranteed  to  no  longer  run  with 
memory  bounded  by  an  sdw,  then  the  store  fault  simulation  can  be  eliminated 
(11  instructions). 


Functional  Description  of  absa/routine 


! 

The  absa  routine  is  an  address  development  package  used  in  coordination 
with  the  various  instruction  simulation  modules  within  the  VMM.  It 
develops  an  18  bit  effective  address  or  24  bit  virtual  final  address  according 
to  the  instruction  being  simulated  and  the  mode  under  which  the  instruction 
would  have  been  executed. 


Usage  Information  for  absa/routine- -This  address  development  routine  is 
invoked  via  a call  to  one  of  the  three  entry  points  as  follows: 


tra  lcdb  | 

tra  lcdb  | 

tra  lcdb  | 

vm/addr/ 1/ entry 
vm/addr/ 2/entry 
vm  / addr  / ea  / entry 

A 24  bit  final  virtual  address  is  constructed  via  a call  to  vm/addr/ 1/ entry 
with  no  index  register  arguments.  The  entry  point  vm/addr/ 2/ entry  is  used 
to  develop  a 24  bit  final  virtual  address  and  it  must  be  called  with  absa / inst 
and  absa/ic/ir  in  the  A and  Q registers  respectively.  The  final  entry 
point  vm/addr/ea/ entry  develops  an  18  bit  virtual  effective  address  and 
requires  no  index  register  arguments. 

_ i 
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Upon  return  to  the  calling  module  the  appropriate  address  will  be  right 
justified  in  the  upper  24  bits  of  the  4 register.  The  contents  of  all  other 
index  register  may  have  been  destroyed. 

Resource  Objectives  of  absa/ routine --Execution  efficiency  should  be  the 
primary  objective  while  memory  usage  shall  be  secondary. 

Design  Overview  of  absa/ routine --The  purpose  of  absa  is  to  simulate  the 
address  development  for  a virtual  instruction's  operand.  This  simulation 
is  performed  by  creating  the  exact  environment  under  which  the  instruction 
would  have  been  executed  in  virtual  mode.  This  includes  restoring  all 
virtual  index  registers,  pointer  registers,  dsbr,  ic  and  the  pertinent 
control  unit  information.  The  processor  mode  is  described  by  the  state 
of  the  following  indicators  as  follows: 

• Zero:  master/ slave.  1 ^ master 

• Overflow:  absolute /append.  1 *>  absolute 

• Exponent  underflow:  6000/6100,  1 $>  6000 

• Negative:  18  bit  effective/  24  bit  final,  1 18  bit  effective 

address 

Once  the  virtual  state  has  been  restored  as  described  above,  the  virtual 
instruction  is  executed  with  the  instruction's  op/code  replaced  by  the  absa 
op/code.  This  absa  instruction  performs  the  appropriate  address  develop- 
ment under  the  control  of  the  aforesaid  indicators.  After  the  construction 
of  the  virtual  address,  control  is  returned  to  the  caller. 
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The  packaging  of  the  absa  routine  is  unlike  most  VMM  routines.  Initially, 
the  routine  is  entered  in  append  mode  in  segment  lcdb.  In  lcdb  the  index 
registers  and  indicators  are  set  up  and  then  control  is  passed  to  the  rpdb  in 
absolute  mode.  At  this  point  the  virtual  dsbr  and  pointer  registers  are 
restored  and  then  a restore  control  unit  is  performed  to  reconstruct  the 
remaining  virtual  conditions  such  as  psr/ppr,  and  ic.  The  restoring  of  the 
control  unit  results  in  the  execution  of  an  xed  pair  containing  the  modified 
absa  instruction  and  a tra.  The  absa  develops  the  virtual  address  while 
the  tra  causes  a return  to  the  VMM  in  rpdb.  The  VMM's  registers  are  then 
restored  before  returning  to  calling  module. 

FUNCTIONAL  DESCRIPTION  OF  DISPATCH 

The  composite  of  modules  comprising  dispatch  is  responsible  for  the 
allocation  of  a real  processor  to  a specified  virtual  processor/ virtual  \ 

machine.  The  control  routine  for  the  dispatching  process  is  invoked  after 
the  VMM  has  completed  the  processing  of  the  exception  (interrupt/fault) 
which  engaged  the  VMM.  At  this  point  of  entry,  the  virtual  processor  which 
was  in  execution  at  the  time  of  the  exception  is  now  ready  to  be  assigned  a 
real  processor. 

However,  before  this  processor  or  any  processor  is  dispatched,  the  VMM 
will  accept  apy  outstanding  interrupts  (the  interrupt  set  directly  by. the  VMM 
is  the  only  exception).  The  dispatch  control  routine  is  the  only  instance 
within  the  VMM  where  an  interrupt  is  permitted.  If  an  interrupt  is  present, 
control  will  be  passed  to  the  fault /interrupt  intercept  module  via  the 
corresponding  interrupt  vector  and  the  interrupt  will  eventually  be  processed 
by  I-proc.  In  the  event  that  no  interrupt  is  outstanding,  the  VMM  will  again 
be  inhibited  from  receiving  interrupts  until  a virtual  processor  is  placed  in 
execution. 


J 
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After  permitting  all  interrupts  to  be  processed,  the  dispatch  control 
routine  will  determine  the  dispatch  state  of  the  virtual  machine /virtual 
processor  receiving  the  initial  exception.  If  this  virtual  processor's 
execution  can  be  suspended,  the  dispatch  module  NEED  will  be  invoked  to 
determine  the  next  virtual  machine /virtual  processor  to  be  dispatched.  The 
conditions  upon  which  a virtual  processor's  execution  may  be  suspended  are: 


1.  The  virtual  processor  is  currently  executing  a "dis" 
instruction. 


2.  The  virtual  processor  was  interrupted  and 


a) 


the  virtual  processor  is  a control  processor 

(i.  e. , the  virtual  processor  can  receive  interrupts) 

and  its  interrupt  masks  are  set  to  allow  interrupts, 

or 


b) 


the  virtual  processor  is  a non-control  processor  and  it 
was  executing  in  slave  mode  when  the  interript  occurred. 


Design  Overview  of  dispatch 


Of  the  two  functions  performed  by  dispatcher/interrupt  recognition  and 
dispatching/ interrupt  recognition  is  handled  first  and  dispatching  is  perform- 
ed later  by  subroutines  of  dispatcher.  Entry  to  dispatch  occurs  after  the 
processing  of  a fault/interrupt  by  F-proc/i-proc.  Upon  entry,  the  scu 
masks  are  set  to  allow  ahy  outstanding  interrupts.  The  occurrence  of  an 
interrupt  results  in  control  being  delivered  to  fiim  via  the  interrupt  vectors. 
After  the  processing  of  the  interrupt,  control  is  eventually  passed  to 
dispatch  where  a test  for  additional  interrupts  will  be  made.  If  no  interrupts 
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are  awaiting  processing,  dispatcher  determines  to  which  eligible  virtual 
processor  it  will  place  in  execution.  The  criteria  for  dispatching  is  as 
follows: 


A.  If  the  VMM  was  entered  via  a valid  VM  fault  (i.  e. , not  a 

fault  occurring  as  a result  of  the  execution  of  a privileged  VMM 
instruction),  the  real  processor  will  continue  to  execute  on  the 
current  virtual  processor. 

B.  If  entry  to  the  VMM  resulted  from  the  attempted  execution  of 
a privileged  VMM  instruction,  then  the  processor  will  be 
dispatched  to  the  same  vp.  This  is  accomplished  by  the  VMM 
routine  vm/  spec/ fault. 

The  dis  instruction  is  an  exception  to  the  above  dispatching  criterion  and 
control  is  not  returned  to  the  same  vp.  Instead  the  module  NEED  is 
called  to  find  an  eligible  vp  for  dispatching. 


Functional  Description  of  vp/int 


This  module  will  cause  the  return  of  a vp  via  a simulated  external 
interrupt  if  there  exists  a pending  external  interrupt --any  interrupt  of  a 
connect  fault.  If  no  external  interrupt  is  pending,  then  the  virtual 
processor  will  resume  execution  at  the  point  where  it  was  initially  inter- 
rupted. 


Usage  Information  for  vp/int-  -When  control  is  given  to  this  module,  x4  must 
contain  dispatch  entry.  Upon  exit  from  this  module  the  vp  will  be  placed 
in  execution. 
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Design  Overview  of  vp/int--This  module  along  with  its  submodules  is  used  io 
return  to  a virtual  processor  whose  execution  has  halted  as  a result  of  an 
interrupt.  As  such  this  module  will  place  the  processor  in  virtual  mode 
via  a simulated  connect  fault,  simulated  interrupt,  or  return  to  the  next 
instruction  following  the  real  interrupt.  The  exact  method  of  return  is 
dependent  upon  the  existence  of  a connect  fault  or  interrupt  which  should  be 
delivered  to  this  virtual  processor.  If  either  condition  exists,  then  return 
would  be  via  the  appropriate  vector.  In  the  event  that  both  exist,  the 
connect  fault  will  have  higher  priority. 


If  there  still  exists  pending  interrupts  after  determining  the  method  of 
return,  these  will  be  queued.  In  order  to  deliver  these  interrupts,  the 
processor's  VMM  interrupt  call  will  be  set  in  the  system  control  unit. 

This  forces  the  virtual  processor  to  be  again  interrupted  when  it  enters  \ 

non -inhibited  code. 


Functional  Description  of  vector / simulation 


Vector /simulation,  as  its  name  implies,  simulates  the  instrvctions  in  an 
interrupt  or  fault  vector  pair.  When  control  is  to  be  returned  to  a vp  by 
means  of  a fault  or  interrupt  this  routine  is  invoked.  For  non-transfer 
type  instructions  in  the  vector  pair,  an  actual  simulation  of  the  instruction 
will  be  performed,  whereas  a transfer  instruction  will  actually  be  executed 
in  the  return  to  the  vp. 


Usage  Information  for  vector/ simulation --In,  addition  to  the  normal 
register  connections,  vector/ simulation  must  be  called  with  the  address 
of  the  vector  pair  relative  to  the  vm  base  in  x3  (vector-base). 
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Resource  Objective  of  vector/ simulation- -It  is  not  the  intent  of  this  module 
to  simulate  the  operation  of  every  6100  instruction.  This  in  itself  would  be 
a major  undertaking  and  althou^i  it  is  necessary  for  the  operation  of  a 
functional  VMM,  it  is  not  practical.  Therefore,  only  the  instructions 
currently  used  by  MULTICS  and  GCOS  in  their  vectors  will  be  simulated. 

At  present  this  set  of  instructions  includes  tsxn,  tra,  rcu,  stcl,  Ida,  ldxn, 
slxn  and  nop.  The  addition  of  other  instructions  would  be  no  more  complex 
than  the  instruction's  simulation. 

Design  Overview  of  vector/simulation--Vector/simulation  is  used  to  simulate 
the  pair  of  instructions  found  in  a vp's  interrupt  or  fault  vector.  As 
mentioned  in  the  resource  objectives,  only  specific  instructions  are 
expected  in  these  pair  and  only  these  will  be  simulated.  A linear  table 
search  1b  used  to  determine  if  the  instructions  in  the  current  search  is  used 
to  determine  if  the  instructions  in  the  current  vector  pair  are  one  of  these  \ jj 

specific  instructions.  At  the  same  time,  the  corresponding  simulation 
routine  will  be  located. 

If  the  instruction  to  be  simulated  is  a transfer  type  instruction,  the  instruc- 
tion is  implanted  into  the  even  instruction  portion  of  the  vp's  control  unit 
block.  The  appropriate  mode  indicators  are  then  set  to  simulate  the  state 
of  a processor  while  executing  a vector  pair.  These  two  operations  will 
result  in  the  proper  execution  of  the  transfer  when  the  control  unit  is 
restored  upon  the  vp's  dispatch. 

For  the  non -transfer  instructions,  the  final  address  of  the  instruction's 
operand  is  developed.  For  MULTICS  this  entails  Invoking  the  absa  routine, 
while  a GCOS  vp  requires  a simple  simulatlQA  for  most  instruction  tags. 
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After  the  final  address  is  developed,  the  instruction  itself  is  simulated. 

After  this  simulation  control  is  returned  to  dispatch. 

Known  Limitations  of  vector/ simulation- -Faults  within  a multi-word 
instruction  in  a vp  without  a transfer  in  the  corresponding  vector  will  result 
in  a VM  abort.  The  same  is  true  for  an  interrupt  within  MULTICS. 

Functional  Description  of  vm/int/test 

Vm/int/test  is  involved  during  the  VMM's  dispatching  algorithm  to  determine 
if  there  exists  an  outstanding  virtual  interrupt  which  the  vp  to  be  dispatched 
can  process. 

Usage  Information  for  vm/int/test- -Entry  to  vm/int/test  is  via  a TSX6 
vm/int/test  and  upon  return  int/type  will  describe  the  interrupt  type  via 
which  the  dispatch  to  the  vp  will  be  made. 

Resource  Objectives  of  vm/int/test- -Under  current  VMM  development,  it  is 
envisioned  that  a vp  will  not  be  receiving  interrupts  from  more  than  one  scu. 
This  is  true  of  the  current  releases  of  both  MULTICS  and  GCOS.  As  such 
the  VMM  will  only  check  the  control  scu  for  outstanding  interrupts. 

Design  Overview  of  vm/int/test- -In  testing  for  outstanding  virtual  interrupts, 
vm/int/test  compares  the  virtual  interrupt  mask  with  the  interrupt  cells  for 
the  vp's  virtual  control  scu.  If  no  match  is  found  between  corresponding 
bits  of  the  mask  and  interrupt  cells  then  control  is  returned  to  the  calling 
module.  However,  if  a match  is  found  and  the  vp  cannot  currently  receive 
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an  interrupt,  then  the  interrupt  cell  corresponding  to  the  current  real 
processor  is  set  in  the  scu  with  the  memory  bank  containing  real  address 
zero.  This  will  cause  this  and  only  this  real  processor  to  be  interrupted  in 
virtual  mode  as  soon  as  non -inhibited  code  is  executed. 

In  the  event  of  a match  and  the  vp  can  be  dispatched  witv  n interrupt  then 
the  entry  for  the  first  such  match  will  be  removed  from  the  virtual  scu's 
interrupt  cell.  If  more  than  one  match  exists,  the  interrupt  cell  correspond- 
ing to  this  real  processor  would  be  set  in  the  scu  with  memory  address  zero. 
Control  will  finally  be  returned  to  the  caller  with  the  type  of  interrupt  that 
the  vp  will  be  dispatched  with. 

Known  Limitations  of  vm/int/ test  --Entry  and  exit  to  vm/int/test  should  be 
gated. 
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SECTION  3 


DESIGN  APPROACH  FOR  VMM  EXTENSION 


The  various  functions  performed  by  VMMs  can  be  segregated  into  two  broad 
categories:  those  which  are  freque./iy  used  and  those  which  are  infrequently 
used.  Some  examples  of  these,  as  a function  of  their  frequency,  are  the 
following: 

• Frequently  used  functions 

--Real  I/O  initiation 

--Fault  and  interrupt  processing 

--Dispatching  of  processors 

--Simulation  of  faults/interrupts  in  virtual  machines 

• Infrequently  used  functions 

--Virtual  machine  definition 

--Virtual  machine  start-up 

--Receipt  and  response  to  virtual  machine  operator  commands 

All  of  these  functions  are  the  same  or  similar  to  functions  which  are  normally 
performed  by  an  operating  system.  This  suggests  that  the  functions  with 
low  use  frequency  needed  in  a VMM  might  be  supplied  by  an  operating 
system  executing  in  one  of  the  virtual  machines,  the  Service  Machine  (SM), 
being  supported  by  the  VMM.  The  primary  advantage  of  this  approach  is 
that  the  code  to  support  such  functions  need  not  be  redundantly  created  and 
maintained  within  the  VMM.  If,  in  addition,  the  functions  supported  by  the 
operating  system  in  the  SM  exist  in  the  form  of  user  (slave)  programs,  some 
important  secondary  advantages  accrue: 
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• The  overall  VMM  functionality  is  (at  a point  in  time)  richer 
because  of  the  relative  ease  of  creating  user  programs. 

• The  incremental  resources  committed  exclusively  to  VMM  are 
relatively  small  because  the  operating  system  in  the  SM 
executes  ordinary  slave  programs  on  behalf  of  ot  her  users  in 
addition  to  providing  service  to  the  VMM. 

• The  currentness  of  the  VMM  with  respect  to  new  devices  is 
maintained  with  relative  ease  by  simply  using  correspondingly 
new  versions  for  the  operating  system  in  the  SM. 

The  service  machine  concept  could  be  used  in  providing  enhanced  functionality 
in  the  VMM.  In  fact,  the  design  effort  on  the  extended  functionality  has 
proceeded  sufficiently  far  to  confirm  the  feasibility  of  the  service  machine 
approach. 

GENERAL  AREAS  OF  FUNCTIONAL  NEED  TO  BE  EXAMINED 

We  wish  to  examine  techniques  for  extending  the  VMM  functionality  to 
include  support  for: 

• Shared  or  dedicated  unit  record  peripherals 

• Shared  front  end  processors 

• Simulated  system  consoles 

• General  user  interfaces 
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The  service  machine  concept  can  be  applied  effectively  when  the  overhead 
incurred  with  this  approach  is  not  significantly  detrimental  to  system 
performance.  Each  of  the  above  designated  functional  areas  has  some 
aspects  which  are  invoked  relatively  rarely  (e.  g. , definitional)  and  others 
which  are  invoked  frequently  (operational).  Thus,  we  would  expect  a 
design  for  these  functions  to  support  some  aspects  via  the  service  machine 
technique  and  others  via  direct  extension  to  the  base  VMM  and  hardware. 

All  of  these  functional  areas  relate  to  the  handling  of  I/O  for  virtual 
machines. 

GENERAL  I/O  CONSIDERATIONS  FOR  VIRTUAL  MACHINES 

We  shall  examine  the  handling  of  I/O  by  the  VMM  and  address  the  subject 
of  the  treatment  of  shared  and  unit  record  peripherals  in  this  section. 

These  considerations  apply  to  all  the  functional  areas  listed  above. 

I/O  Device  Specification 

The  reference  to  a device  in  virtual  machine  I/O  needs  to  be  mapped.  The 
VMM  has  for  each  virtual  machine  a table  of  resources  that  can  be  consulted 
to  determine  the  actual  support  being  used  for  the  virtual  machine  devices. 
Distinct  forms  that  this  support  might  assume  are  discussed  in  detail 
below  in  the  section  on  virtual  device  support. 

In  the  simplest  form  of  device  support,  the  VMM  maps  each  virtual  machine 
device  into  some  real  device  of  the  same  type.  The  VMM  then  merely 
substitutes  references  to  the  real  device  for  references  to  the  virtual 
device  in  the  I/O  operations. 
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I/O  Program  Analysis 


The  approach  to  mapping  memory  addresses  and  device  references  is 
actually  more  complex  than  described  above.  First,  the  mapped  form  of 
the  I/O  program,  with  the  virtual  addresses  replaced  by  real  addresses, 
cannot  appear  in  the  addressable  memory  of  the  virtual  machine.  If  it  did, 
other  elements  of  the  VM,  such  as  the  virtual  processor,  could  reference 
the  I/O  program  and  read  or  alter  absolute  addresses.  Therefore,  the 
VMM  must  analyze  the  virtual  machine  I/O  program  and  construct  a trans- 
lated I/O  program  in  a work  area  private  to  the  VMM. 

Second,  the  translated  I/O  program  may  not  be  the  same  size  as  the 
virtual  I/O  program.  For  example,  in  a machine  using  a paging  mechanism 
for  memory  mapping,  a contiguous  region  of  virtual  memory  need  not  map 
a contiguous  region  of  real  memory.  Thus,  I/O  commands  that  involve 
data  transfers  crossing  page  boundaries  might  have  to  be  split  into  multiple 
commands  in  a real  I/O  program.  Again,  the  VMM  must  construct  the 
real  1/O‘program  in  a private  work  area. 

A third  complication  arises  in  dynamically  modifiable  I/O  programs.  The 
central  processor,  the  I/O  program  itself,  or  even  some  other  simultaneously 
operating  1/ O program  may  attempt  to  alter  the  virtual  machine  I/O  program 
during  its  operation.  To  reflect  this  functionality  of  a real  machine  fully 
would  require  a complex  and  large  addition  to  the  VMM  and  would  introduce 
significant  overhead  processing  when  invoked.  Third -generation  VMMs  in 
general  reflect  the  conclusion  that  providing  the  full  functionality  is  not 
required  in  a practiced  sense. 
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Any  dynamic  modification  to  an  I/O  program  must  be  checked  by  the  VMM  to 
verify  that  the  new  form  of  the  program  will  reference  only  resources 
assigned  to  the  virtual  machine.  By  not  supporting  dynamic  modification 
of  virtual  machine  I/O  programs,  it  is  considerably  easier  to  ensure  the 
integrity  of  the  VMM  and  all  virtual  machines.  Since  a real  I/O  program 
is  constructed  in  a work  area  private  to  the  VMM,  it  cannot  be  modified  by 
the  virtual  machine's  processor  or  by  any  other  I/O  program  of  the 
virtual  machine. 

Caution  must  be  observed  when  moving  operating  system  software  from  a 
virtual  machine  to  a real  machine.  Since  dynamically  modified  I/O 
programs  may  behave  differently  on  virtual  and  real  machines,  it  is  possible 
to  develop  software  that  operates  correctly  on  a virtual  machine  but  fails 
on  a real  system. 

I/O  Summary 

In  summary,  basic  third -generation  virtual  machine  I/O  operations  are 
handled  in  the  following  sequence.  The  processor  attempts  execution  of 
the  instruction  (cioc)  to  start  an  I/O  processor  executing  an  I/O  program. 
This  causes  a trap  to  the  VMM,  which  analyses  the  I/O  program,  maps 
memory  addresses  and  device  references,  and  creates  a real  I/O  program 
in  a work  area  private  to  the  VMM.  The  VMM  then  queues  the  real  I/O 
program  for  execution  by  the  real  I/O  processor.  When  execution  of  the 
I/O  program  is  completed,  an  interrupt  to  the  central  processor  occurs. 

This  causes  control  to  return  to  the  VMM,  which  analyses  the  case  of  the 
interrupt.  If  the  interrupt  signified  I/O  completion  for  some  particular 
virtual  machine,  the  VMM  simulates  the  occurrence  of  an  interrupt  for 


the  virtual  machine  by  manipulating  the  saved  state  information  of  the 
virtual  machine.  The  VMM  then  makes  a dispatching  decision  taking  into 
consideration  the  interrupt  event.  Thus,  the  software  on  a virtual  machine 
receives  notification  of  IP  program  completion  precisely  as  it  would  on 
a real  machine. 

VIRTUAL  DEVICE  SUPPORT 

Many  applications  for  VMs  depend  on  the  ability  of  the  VMM  to  provide  an 
appropriate  type  of  virtual  device  support.  Figure  1 illustrates  four 
different  categories  of  support:  dedicated,  partitioned,  mapped,  and 
simulated.  The  mapping  of  device  names  and  device  data  addresses,  that 
is,  addresses  of  data  objects  on  devices,  is  discussed  below  for  each  of 
these  support  categories. 

Dedicated  Support 


In  the  dedicated  mode  of  support,  the  VMM  maps  a device  of  the  virtual 
machine  into  an  identical  device  of  the  real  machine.  No  other  virtual 
machines  are  allowed  any  form  of  access  to  the  real  resource.  An  example 
would  be  a disk  drive  assigned  to  a particular  virtual  machine.  The 
particular  device  name  may  be  different  in  virtual  and  real  machines  and  is 
mapped  by  the  VMM. 

Partitioned  Support 


In  partitioned  support,  the  data  addresses  on  the  virtual  device  correspond 
directly  to  those  on  the  assigned  real  device;  unlike  dedicated  support. 


however,  the  real  device  need  not  be  identical  to  the  virtual  device.  For 
example,  as  shown  in  Figure  1,  a larger  real  disk  can  be  used  to  support  a 
smaller  virtual  disk. 


Mapped  Support 

Most  VM  devices  are  supported  in  the  mapped  mode.  This  means  that  data 
addresses  are  mapped  via  some  simple  transformation  into  real  addresses. 
The  mapping  is  typically  a base-bounds  form  although  a multi-area  form,  as 
shown  in  Figure  1,  might  be  used. 

Simulated  Support 

In  simulated  support  the  VMM  software  plays  an  important  role  in  creating 
the  functionality  visible  to  the  virtual  machine.  This  mode  can  be  applied 
to  any  type  of  resource  and  is  used  for  many  reasons: 

• The  resource  does  not  exist  in  reality  and  hence  must  be 
simulated; 

• Unit  record  devices  must  be  spooled  using  disk  or  tape; 

• A virtual  machine  does  not  need  the  full  capacity  of  its  virtual 
devices  and  so  the  VMM  compacts  many  such  devices  onto 
some  other  kind  of  device;  and 


• A virtual  machine  needs  to  address  more  of  a resource  than 
actually  exists  and  the  VMM  simulates  the  resource  using  other 
resources. 


VMM  Involvement 


For  each  of  the  modes  of  support  for  I/b  equipment,  the  VMM  has  a 
different  degree  of  involvement.  However,  the  basic  idea  of  having  control 
pass  to  the  VMM  at  the  start  of  an  I/O  operation  and  the  eventual  interrupt 
reflection  by  the  VMM  back  to  the  virtual  machine  remains  unchanged  in  all 
modes  of  support  in  third -generation  VMMs.  The  VMM  support  required 

when  simulating  a resource  is  greater  than  for  othet  modes,  and  can  be 
arbitrarily  complex  depending  on  the  definition  of  the  functionality  of  the 
resource. 

SPECIAL  CONSIDERATIONS  FOR  FRONT  END  PROCESSOR  FUNCTIONAL 
EXTENSIONS 

The  support  of  the  front-end  processor  (DN)  could  be  accomplished  in 
several  ways  some  of  which  require  hardware  modification  to  the  equipment 
(DN355,  etc.). 

The  DNs  internally  use  a line  numbering  scheme  by  which  they  identify  the 
source  or  destination  of  messages.  Clearly,  these  line  numbers  need  to  be 
related  to  the  line  identification  used  within  an  operating  system  in  a VM. 

The  relationship  between  DN  line  numbers  and  operating  system  line  numbers 
could  be  maintained  within  the  VMM.  In  this  situation  a fairly  heavy  burden 
of  processing  would  fall  on  the  VMM  for  a highly  interactive  or  transaction 
processing  environment  since  each  transmission  in  or  out  would  require 
software  I/O  interpretation  of  line  addresses.  An  alternate  approach  is  to 
incorporate  modifications  to  the  DN  hardware  and  software  which  make  the 
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DN  aware  of  the  VMM.  In  this  case,  the  DN  can  be  used  to  manipulate  the 
line  addresses.  This  approach  is  in  effect  placing  part  of  the  VMM  in  the 
DN  for  mapping  real  to  virtual  resources.  The  DN  needs  to  be  aware  of 
the  identification  of  the  VM  for  which  it  is  performing  service.  This  can 
be  accomplished  by  a mechanism  for  the  VMM  in  the  main  machine  to  talk 
to  ita  counterpart  in  the  DN.  In  effect,  the  two  VMM  parts  would  identify 
to  each  other,  for  each  transmission,  which  VM  was  involved. 

The  issue  of  secondary  storage  also  needs  to  be  addressed  in  the  context  of 

' 

front  end  processors.  The  DN  and  the  main  machine  can  share  secondary 
storage  equipment  and  exchange  information  via  records  stored  on  the 
equipment.  Certainly,  for  VM  integrity  considerations,  the  DN  must  not 
be  permitted  global  access  to  the  shared  secondary  storage.  Even  if 
access  were  permitted  to  that  part  of  the  secondary  storage  corresponding 
to  a particular  virtual  machine,  j,  the  DN  software  would  have  to  be  designed 
to  use  the  appropriate  line  identifications  to  avoid  confusing  the  VM 
operating  system. 

The  only  currently  known  technique  for  supporting  shared  secondary  storage 
between  DN  and  associated  VM  is  to  incorporate  a VMM  internal  to  the  DN 
hardware  so  that  the  DN  software  for  a particular  VM  uses  the  same  line 
number  identification  as  the  VM.  Although  we  have  been  discussing  only 
line  number  identification  information,  other  characteristics  of  communica- 
tions hardware  also  need  proper  virtualization.  For  example,  different 
types  of  real  terminals  might  be  used  instead  of  the  type  supported  within 


the  VM.  In  this  case,  the  VMM  within  the  DN  hardware  could  simulate  the 
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SPECIAL  CONSIDERATIONS  FOR  SYSTEM  CONSOLES 


It  would  be  possible  to  use  any  terminal  equipment  as  a system  console  with 
proper  VMM  and  Service  Machine  support;  in  particular,  using  terminals 
supported  under  TSS  in  a SM  using  GCOS. 


The  system  console  is  conceptually  all  the  buttons,  switches,  lights,  ^nd 
keyboard  available  to  the  operator  of  a real  system;  To  that  extent,  all 
operator  functions  need  to  be  supported  via  the  virtual  machine's  operator 
console. 


One  approach  to  supporting  the  console  is  to  define  modes  of  terminal  usage. 
One  mode  is  that  in  which  the  terminal  behaves  like  the  keyboard  and  printer 
of  a real  console.  The  second  mode  is  that  in  which  the  terminal  emulates 
the  lights,  buttons,  and  switches.  There  is  also  a third  mode  of  interest. 
This  is  a mode  to  support  functions  not  available  on  a real  computer  system. 
An  example  of  this  third  mode  is  the  function  of  interactively  defining  a 
new  virtual  machine  configuration  to  be  catalogued  in  a library  of  VM 
definitions. 


The  main  technical  issue  to  support  these  modes  is  that  there  needs  to  be 
a mechanism  to  easily  shift  among  modes,  such  as  typing  special 
characters. 


An  additional  special  consideration  for  consoles  is  that  messages  from  the 
operating  system  in  the  virtual  machine  will  use  the  virtual  device 

designators.  The  mapping  between  virtual  and  real  could  be  done  by  special 

* 

recognition  processing  for  that  operating  system  and  run  in  the  service 


machine  so  that  when  messages  appeared  on  the  terminal,  they  would  con- 
tain real  device  designators.  Alternatively,,  the  operator  could  use  special 
commands  to  map  between  virtual  and  real  designators. 

The  support  of  system  consoles  is  a slow  speed  activity  compared  to  other 
device  1.0  and  therefore  fits  well  into  the  service  machine  approach.  Also, 
the  third  mode  of  use  for  system  consoles  fits  well  with  the  service 
machine  concept  since  that  mode  needs  several  operating  system  services 
such  as  the  memory  capability  of  the  file  system  for  storing  virtual  machine 
definitions. 


\ 

\ 


